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NUMERICAL CRITERIA FOR VERY AMPLENESS 
OF DIVISORS ON PROJECTIVE BUNDLES 
OVER AN ELLIPTIC CURVE 

ALBERTO ALZATI, MARINA BERTOLINI AND GIAN MARIO BESANA 



ABSTRACT. Let D be a divisor on a projectivized bundle over an elliptic curve. 
Numerical conditions for the very ampleness of D are proved. In some cases a complete 
numerical characterization is found. 

1. Introduction. Ampleness of divisors on algebraic varieties is a numerical prop- 
erty. On the other hand it is in general very difficult to give numerical necessary and 
sufficient conditions for the very ampleness of divisors. In [4] the author gives a suffi- 
cient condition for a line bundle associated with a divisor D to be normally generated 
on X = P(E) where £ is a vector bundle over a smooth curve C. A line bundle which 
is ample and normally generated is automatically very ample. Therefore the condition 
found in [4], together with Miyaoka's well known ampleness criterion, gives a suffi- 
cient condition for the very ampleness of D on X. This work is devoted to the study 
of numerical criteria for very ampleness of divisors D which do not satisfy the above 
criterion, in the case of C elliptic. With this assumption Biancofiore and Livomi [3] (see 
also [2, Proposition 8.5.8] for a generalization) gave a necessary and sufficient condition 
when E is indecomposable, rk£ = 2 and deg£ = I. Gushel [6] also gave a complete 
characterization of the very ampleness of D assuming that E is indecomposable and \D\ 
embeds A' as a scroll. This work deals with the general situation and addresses the cases 
still open. 

The main technique used here is a very classical one. A suitable divisor A on A' is 
chosen such that there exists a smooth S G \A\ containing every pair of points, possibly 
infinitely near. Appropriate vanishing conditions are established to assure that the natural 
restriction map H°(X, Ox(D)) — » H°(S, Ox(D)\s) is surjective. In this way we get that a 
divisor D of X is very ample if and only if D\s is very ample. In this context S is chosen 
as S = P(E') where E' is a quotient of E, thus with rank smaller than rank E. Therefore an 
inductive process on the rank can be set up. This process is not always easy to carry on. 
For example if E is assumed to be indecomposable there is no guarantee that E' will still 
be indecomposable. Since ampleness is inherited by quotients, we will require at some 
stage that E be ample. The paper is organized as follows. Section 2 contains notation, 
known and preliminary results used in the sequel. In Section 3 the case of rank E = 2 is 
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fully treated. We recover Biancofiore and Livorni's results and deal with the case of E 
decomposable. Section 4 deals with the case of rank £ = 3 while Section 5 contains the 
study of case rank E > 4. In particular in the case of rank E = 3 we get the following 
result (see Section 2. 1 for notation): 

THEOREM . Let Ebea rank 3 vector bundle on an elliptic curve C and letD = a T+ bf 
be a line bundle onX= P(£). 

(a) If E is indecomposable then 

(a I) if d = 0 (mod 3), D is very ample if and only ifb + a^~(E) > 3, 
(a2) if d = 1 (mod 3), D is very ample ifb + an~(E)>l, 
(a3) ifd = 2 (mod 3), D is very ample ifb + ap~(E) > f , 

(b) ifE is decomposable, then D is very ample if and only ifb + ap.'{E) > 3 except 
when E = £, ©£ 2 , with rk£, = I, rk£ 2 = 2, deg£ 2 oddanddegE x > In 
the latter case the condition is only sufficient. 

Notice that the above theorem shows the existence, among others, of a smooth threefold 
of degree 20 embedded in P 9 as a fibration of Veronese surfaces over an elliptic curve, 
choosing a - 2, b = - 1 and d = 4. 

The authors would like to thank Enrique Arrondo and Antonio Lanteri for their 
friendly advice. The third author would like to thank the Department of Mathematics of 
Oklahoma State University for the kind hospitality and warm support during the final 
stages of this work. 

2. General results and preliminaries. 

2.1. Notation. The notation used in this work is mostly standard from Algebraic Geom- 
etry. Good references are [9] and [5]. The ground field is always the field C of complex 
numbers. Unless otherwise stated all varieties are supposed to be projective. P" denotes 
the w-dimensional complex projective space and C* the multiplicative group of non zero 
complex numbers. Given a projective n-dimensional variety X, Ox denotes its structure 
sheaf and ?ic(X) denotes the group of line bundles over X. Line bundles, vector bundles 
and Cartier divisors are denoted by capital letters as L, M, — Locally free sheaves of 
rank one, line bundles and Cartier divisors are used interchangeably as customary. Let 
L, M e ?ic{X), let £ be a vector bundle of rank r on X, let f be a coherent sheaf on X 
and let Y C X be a subvariety of X. Then the following notation is used: 

LC the intersection number of L with a curve C, 

L" the degree of L, 

|L| the complete linear system of effective divisors associated with L, 

L\y the restriction of L to Y, 

L ~ M the linear equivalence of divisors, 

L = M the numerical equivalence of divisors, 

Num(A) the group of line bundles on X modulo the numerical equivalence, 

£* the dual of £, 

P(£) the projectivized bundle of £, 
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£T(X, J) the /-th cohomology vector space with coefficient in JF, 
/»'(*, the dimension oiH(X, J). 
If C denotes a smooth projective curve of genus g, and E a vector bundle over C of 
deg£ = C|(£) = d and rk£ = r, we need the following standard definitions: 

E is normalized \ih\E) f 0 and h°(E ® Z,) = 0 for any invertible sheaf £, over C 

with degZ, < 0. 

E has slope /i(£) = ^. 

£ is semistable if and only if for every proper subbundle 5, /x(5) < /i(£). It is 

stable if and only if the equality is strict. 

The Harder-Narasimhan filtration of E is the unique filtration: 

0 = £o C £, C • • C & = £ 

such that ^ is semistable for all /, and Hi(E) = /x(^) is a strictly decreasing 
function of i. 

We recall now some definitions from [4] which we will use in the following: let 
0 = E 0 C E\ C • • • C E t = E be the Harder-Narasimhan filtration of a vector bun- 
dle E over C. Then 

//-(£) = fl s (E) = 

^{E) = n ] (E) = fi(E l ), 
or alternatively 

= max{/x(5) | 0 — » S — ► £}, 
= min{/i(0 | £ Q — 0}. 
We have also > > pT{E) with equality if and only if £ is semistable. In 
particular if C is an elliptic curve, an indecomposable vector bundle E on C is semistable 
and hence n(E) = yT(E). Moreover if F, G are indecomposable and hence semistable 
vector bundles on an elliptic curve C and F — ► G is a non zero map, it follows that 
/x(F) < /i(G). 

2.2. General results. Let C be a smooth projective curve of genus g, E a vector bundle 
of rank r, with r > 2, over C and tt:X = P(£) — > C the projective bundle associated to 
£ with the natural projection x. With standard notations denote with T = C>(f)(l) the 
tautological sheaf and with = tt* O c (P) the line bundle associated with the fiber over 
PeC. Let T and / denote the numerical classes respectively of T and !Fp. 

Let D ~ aT + tt*5, with a £ 1, B e Pic(C) and degfl = fc, then D = aT + bf. 
Moreover tt*D = S°(E) ® CH#) and hence /i~(7r.D) = a/i~(£) + b (see [4]). 

Regarding the ampleness, the global generation, and the normal generation of D, the 
following criteria are known: 

THEOREM 2.1 (MlYAOKA [10]). Let E be a vector bundle over a smooth projective 
curve C of genus g, and X = P(E). IfD = aT+bfisa line bundle overX, then D is ample 
if and only if a >0andb + apr{E) > 0. 

Proposition 2.2 (Gushel [7], Proposition 3.3). Let D ~ a¥ + n*B where a>0 
and B G Pic(C), be a divisor on a projective bundle ir.X= P(E) — * C. Then: 
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i) ifa = I, the bundle n*(D) is generated by global sections if and only if the divisor 
Dis, 

H) if <* > 2, and the vector bundle 7r,(Z)) is generated by global sections, then also 
the divisor D is. 

LEMMA 2.3 (GUSHEL [6], PROPOSITION 3.2). Let E be an indecomposable vector 
bundle over an elliptic curve C. E is globally generated if and only if deg E > rank£. 

LEMMA 2.4 (SEE e.g. [4], LEMMA 1.12). Let Ebea vector bundle over C of genus g. 

i) ifp,-{E) >2g-2 then h\C, E) = 0, 

ii) iflT(E) > 2g- 1 then E is generated by global sections. 

For the following theorem we need a definition: 

Definition 1 (Butler [4]). Let £ be a vector bundle over a variety Y, and let 
tt: X = P(E) -* Y be the natural projection. A coherent sheaf 7 over X is said to be 
tir-regular if, for all i > 0, 

-i))=0. 

THEOREM 2.5 (BUTLER [4]). Let E be a vector bundle over a smooth projective 
curve C of genus g, and X = P(E). If D is a (— 1 )n-regular line bundle over X, with 
p,~(ir m D) > 2g, then D is normally generated. 

REMARK 2.6. Let D be a divisor of X = P(£), with E vector bundle on a smooth 
projective curve of genus g. As tt [ffp^D^^—X — /')) =0for/> l,the(— l)7r-regularity 
of D is satisfied, hence the condition an~(E) + b > 2g implies that D is normally 
generated. If a line bundle D on a projective variety A' is ample and normally generated it 
is very ample. Hence from Theorem 2. 1 and 2.5 we get that D is very ample on X = P(ZT) 
if 

(1) b + an-(E)>2g. 

Hence, if g = 1 , the very ampleness ofD = aT+bf is an open problem only in the range 

(2) 0<6 + a/i-(£)<2. 

2.3. Preliminaries. The following result is standard from the theory of vector bundles 
(see [9]): 

LEMMA 2.7. Let E be an indecomposable vector bundle of rank r on an elliptic curve. 
IfE is normalized then 0 < deg E <r — 1 . 

LEMMA 2.8. Let E = £, be a decomposable vector bundle over an elliptic curve 
C, with Ej indecomposable vector bundles. Then n~(E) = min /*(£,•). 

PROOF. For the proof we need the following three claims. 

CLAIM 1 . Let E - ©, £, be as above, then /i(£) > min p(E,). 
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PROOF. Let us denote by r = rk(£), r, = rk(£,), d - deg(£), d { = deg(£,). Let us 
consider the vectors v, in R whose coordinates are (r,, </,•) and the vector v = £, v,. Let 
a, be the angle between the r-axis and v,. Let or be the angle between the r-axis and v. It 
is n(E) =*= tg(or) > min, tg(or ( ) = min,(£) = min, p{E t ). ■ 

CLAIM 2. Let E = ©,£, be as above, and /x(£,) = * = h € Q, for all i. Then 
fi-(E) = h. 

PROOF. Notice that under this hypothesis /i(£) = h. Moreover, by definition, it 
is ii~{E) = min{/i(0 | E -> Q -» 0}. If Q is decomposable in the direct sum of 
indecomposable vector bundles Q k , the existence of a surjective map E -+ Q — ► 0 
implies the existence of surjective maps £—►£?*—► 0 for all k and consequently from 
Claim 1, /z~(£) = min{/i(0 | E — > — + 0, and Q indecomposable}. 

Now let Q 0 be an indecomposable vector bundle which realizes the minimum, i.e. 
H(Q 0 ) = H~(E). From ©,•£/ — » Q 0 — * 0 it follows that there exists at least an index z'o 
such that the map E io — » Q is not zero and /x(£/ 0 ) < /i(0 o ). Therefore it is h < n~(E). 
Ash> n~(E), the claim is proved. ■ 

Claim 3. Let E = ©, £, be as in Claim (2). Then E is semistable. 

Proof. It is enough to prove that for any S vector bundle on C such that there exists 
a map 0 — ► S — » E then /i(S) < /i(£) = h. If we consider the dual map E* — » 5* — ► 0 we 
have /i(5**) > n~(E*) = /!"(©, £/») and, as /i(£*,) = — ^ = —h, from Claim 2 applied to 
E* we have n~(E*) = -A. Hence = ~n(S) > -h and n(S) < h. ■ 

The lemma can now be proved. 

Let E = ©/ Ei be as in the hypothesis of the lemma, and denote by /i, = p(E f ). We 
can choose an ordering such that E = E\ © E 2 ® £3 • • • and m > pi > m • ■ ■ . Let 
£ = (BjL^, be a new decomposition of £ such that each A k is an indecomposable vector 
bundle or a sum of indecomposable vector bundles £, with the same /x, . In this way 
we get a strictly decreasing sequence n(A\) > n(A 2 ) > • > n(A s ), and by claim (3) 
each Ak is semistable. Moreover the sequence 0 C F| CF2 C ••• CF, = £ with 
Fi f = A 1 ©^2 © • • • SM» with 1 < /' < s, is the Harder-Narasimhan filtration of £ because 
the sequence of the slopes = is strictly decreasing and each ^- = Aj is 

semistable for all /' = 1 • • s. Hence we get /x"(£) ■ Mj^) = = min /*(^ <)• ■ 

LEMMA 2.9. Ler D ~ aT + tt*5 fte a /ine fcu/irf/e m X = P(£) over a cwrve C of genus 
g = 1, m7/i 5 G Pic(C), a > 1 anddegB = b. 

i) If a= 1 D is globally generated if and only ifb + n~(E) > 1. 

ii) Ifa>2D is globally generated ifb + ay.~{E) > 1. 

PROOF. To prove ii) it is sufficient to apply Proposition 2.2 and Lemma 2.4. To prove 
i) notice that if £ is indecomposable it is enough to apply Proposition 2.2 and Lemma 2.3, 
observing that an indecomposable vector bundle £ over an elliptic curve is semistable and 
hence /!"(£) = n(E). Let now £ be decomposable and hence E®B decomposable over C. 
In particular let £® B = ©^, A q be a decomposition of £® B in indecomposable vector 
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bundles A q over C. By Lemma 2.3 every A q , for q = 1 , « • • , s, is globally generated if and 
only ifdeg^ > rkA q ,i.e. if and only ifn(A q ) > 1, for all q. From Lemma 2.3, Lemma2.8 
and Proposition 2.2 we get the following chain of equivalences which conclude the proof: 
H~(E) + b > 1 <=> ® fl) = min^ p(A q ) > 1 /z^) > 1 for all q & A q is globally 
generated for all q & n*D is globally generated on C D is globally generated on X. m 

The above lemma is partially contained in [6, Proposition 3.3]. Unfortunately the proof 
presented there is based on [6, Proposition 1 . l(iv)], which is not correct, as the following 
counterexample shows. Let E be an indecomposable vector bundle over an elliptic curve 
with deg£ = 1 and rank£ = 2. Then IT = Q»(E)(2) is generated by global sections, 
according to [2, Proposition 8.5.8]. On the other hand let -k,(2T) = S 2 E = © ? A q , where 
A q is indecomposable for all q. Then &E is generated by global sections if and only if A q 
is such, for all q. From Lemma 2.3 it follows that &E is globally generated if and only if 
n(A q ) > 1 for all ?,/.<?. if and only if n~(S 2 E) > \,i.e. if and only if2/i~(£) = 2p{E) > 1 
which is false. 

If we consider an indecomposable vector bundle of degree d = 0, we have the 
following proposition. It is contained in [7, Theorem 3.9], but we prefer to give here a 
simpler proof. 

PROPOSITION 2.10. Let E be an indecomposable rank r vector bundle over an elliptic 
curve C with deg£ = 0 (mod r), and let D = aT + bf be a line bundle on X = P(£). 
Then D is very ample on X if and only ifb + ay.~{E) = b + ap,(E) > 3. 

Proof. It is enough to consider the case in which E is normalized, as if £ is not 
normalized we can consider its normalization E = E® L with degZ, = /. If D = aT + bf 
in NumP(£) then in NumP(£) we get 

(3) D = aT+(b-al)f, ~d = degE = d + rl, ^E) = n(E) + al. 

Let E be normalized, hence d = 0 and E = F, in the notation of [1] (recall that F\ = Oc). 
According to (2) the only cases to be considered are b = 1 and£ = 2 and hence I) = aT+f 
or D = aT+ If. We want to show that in both these cases D is not very ample. Assume 
the contrary and proceed by induction on r. Let r = 2. As DT = 1 or 2, the smooth elliptic 
curve T, which is the only element of |T|, is embedded by <f>\ D \ as a line or a conic which 
is a contradiction. Assume now the proposition true for F r -\ and recall that there is a 
short exact sequence (see [1] p. 432) 

(4) 0-»O c — F r — F r _, -+0. 

Let V = T\ Y and/ =f\ Y the generators of Num(10 where Y = P(F r _,) c X = P(E). If 
D is very ample, D\y is very ample too; but D\y = aT + bf and it is not very ample by 
induction hypothesis. Hence D is very ample if and only if b > 3. ■ 

The following lemma, which gives a sufficient condition for the very ampleness of a 
divisor DonX= P(£), will be needed later on. 
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LEMMA 2. 1 1 . Let E be a rank r vector bundle over a curve C and let D = aT+bf be 
a line bundle onX = P(£), with a > 1 . Ifn,D is a very ample vector bundle on the curve 
C, then D is very ample on P(E). Moreover if a= 1, D is very ample on X if and only if 
7r»D is very ample on C. 

PROOF. We give only a sketch of the proof. A divisor D = aT+bf on X defines a 
map </?| D | in a suitable projective space such that X = <P\d\(X) is a bundle on C whose 
fibers are the Veronese embedding of the fibers of A' = P(£). Moreover each fiber of A* 
is embedded in a fiber of the projective bundle P(S°(E) ® O c (B)). It follows that the 
very ampleness of 5°(£) ® OdB) and hence of its tautological bundle implies that the 
map <p\ D \ gives an embedding and hence that D is very ample. The case a = 1 follows 
immediately from the above considerations. ■ 

2.4. The case a = 1. We want to investigate the very ampleness of D = aT+ bf in 
dependence of a and b. As we have remarked at the end of Section 2.2, the problem is 
open only when 0 < b + ap,~(E) < 2. Let us begin with the case a = 1 . In this case we 
have the following theorem: 

Theorem 2. 12 (Gushel [6], Theorem 4.3). LetD ~ T + ir*B be a divisor on P(E), 
where E is an indecomposable and normalized vector bundle of rankr over an elliptic 
curve C. Ifb = degfl, the divisor D is very ample if and only if: 

i) b > 3i/deg£ = 0, 

U) b>2ifO<degE<r. 

Now it is easy to prove the following (see (3)): 

PROPOSITION 2.13. In the above assumptions and notations, ifE is indecomposable 
but not normalized, it follows that D is very ample if and only if the following conditions 
hold: 

i) b + /i(£) >3ifd = 0 (mod r). 

ii) b + n(E) > 2 otherwise. 

Remark 2. 14. The previous results consider the case in which E is indecomposable. 
IfE is decomposable, by Lemma 2.12, we can argue as follows: firstly in this case, as 
a = 1,D is very ample if and only if 7r„(£>) is very ample. Secondly we have D ~ T+7r*5, 
tt,(D) ~ £<g> O c (B) = © £ y ® OdB), with Ej indecomposable vector bundles. Moreover 
E <g> O c (B) is very ample if and only if every Ej <g> O c (B) is very ample. Let deg£) = dj 
and rk£ ; = ij, and assume that dj = 0 (mod r y ), possibly only fory = • • /. Then D is 
very ample if and only if b + ^ > 3 fory = 1 • • • t, and b + ^ > 2 for the remaining/s, 
by Proposition 2.13. 

Having dealt above with the case a = 1, from now on the blanket assumption a > 2 
will be in effect. 
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3. Rank 2. Let E be a rank 2 vector bundle on an elliptic curve C and let D = aT+ bf 
be a line bundle on X = P(E). Assume that E is indecomposable. If E is normalized then 
deg£ = 0, 1 by Lemma 2.7. If deg£ = 0, from Proposition 2.10 it follows that D is very 
ample if and only if b > 3. If deg£ = 1, necessary and sufficient conditions for the very 
ampleness of D are given by the following theorem, reformulated under our assumption 
that a > 2. 

Theorem 3.1 (Biancofiore-Livorni [3], Theorem 6.3). Let D~ aT + n*B be a 
divisor on P(E), where E is an indecomposable normalized vector bundle of rank 2 and 
degree 1 over an elliptic curve C. Ifb = deg B, the divisor D is very ample if and only if 
Z>+§ >1. 

The following proposition can now be easily proved (see (3)). 

PROPOSITION 3.2. In the above hypothesis, ifE is indecomposable but not normalized, 
D is very ample if and only if the following conditions hold: 
ii) b + at x-(E) >3ifd = 0 (mod 2). 
i) b + ap-(E) >\ifd=\ (mod 2). 

The case E decomposable is treated by the following theorem. 

THEOREM 3.3. Let D ~ al '+ tt*B be a divisor on P(£), where E is a decomposable 
vector bundle of rank 2 over an elliptic curve C, b = deg B. The divisor D is very ample 
if and only ifb + an~(E)>3. 

Proof. To prove the sufficient condition let E be decomposable as //© G where H 
and G are line bundles on C with deg// = h > deg G = g. By Lemma 2.8 it is n~(E) = g. 
By Lemma 2. 1 1 , a sufficient condition for the very ampleness of D on X is that 7r,(D) is 
very ample as a vector bundle on C. In our hypothesis 

7T,(D) = S°(E) ® Oc(5) = 0 //^ ® G® a ~ q 8 B. 

Now 7T»(D) is very ample if each element of its decomposition has degree > 3, i.e. if 
qh+(a - q)g+b > 3, for all q = 0, . . . , a. As the minimum of qh+(a - q)g+b is realized 
for q = 0, 7r,(D) is very ample if and only if ag + b = b + ap.~(E) > 3. This condition 
is also necessary for the very ampleness of D. Indeed the projective bundle P(G), by the 
exact sequence 

0 — //-* £-^G->0 

[6], Proposition 1.1, gives an elliptic curve r on X, T 6 \T + n*(H*)\. Notice that 
/r°(*,T + 7r* (//*)) > 0. IfDisveryampleitmustbeDr = (ar+6/Xr-/i/) = ag + !> = 
6 + a/|-(E)>3. ■ 
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4. Rank 3. In this section and in the next one, we will prove the very ampleness 
of a divisor on a smooth variety following a classical method, based on the following 
lemmata. 

LEMMA 4. 1 . Let X be a smooth variety, D a divisor in Pic(X) and let A be an other 
element of?'\c{X), such that h x (X, O x (D - A)) = 0. If, for each pair of points R,Q eX 
(possibly infinitely near) it is possible to find a smooth element S G \A \ containing R and 
Q, then D is very ample on X if and only ifD\$ is very ample on S. 

PROOF. If D is very ample then obviously D\s is very ample. On the other hand, 
pick any two points R,Q € X (distinct or infinitely near) and choose S € \A\ such that 
R,Q eS. As D\s is very ample there exist sections of D\s separating R and Q. Now look 
at the following exact sequence 

0— ► Ox(D-A)—> QxiD) -» Os(D\ s ) - 0. 

From the assumptions above we get that the map H°(X, Ox{D)) -» H°(S, O s (D\ s j) is 
surjective and hence D is very ample on X if and only if D\ s is very ample on S. m 

LEMMA 4.2. Let E be an ample vector bundle over an elliptic curve C such that 
deg£ < rk£. Let X = P(£). let Pbea fixed point ofC, D ~ aT+ n*B and A = H+ 7 P be 
line bundles on X, withb = degfi, b- \+(a-\ )/!"(£) > 0andh°(X, O x (A)) > deg£+3. 
Then the hypothesis of Lemma 4. 1 are satisfied for A. 

PROOF. If (a - l)/i-(£) + b - 1 > 0 then by Lemma 2.4 it is h x (X, OKD - A)) = 0. 
Moreover being h°(X, O x {A)) > deg£ + 3, for each pair of points /?, Q € X there exists 
a linear subsystem L C with dim L > deg £, all the elements of which contain R 
and Q. Moreover in L there is at least one smooth element S. In fact, assume that all 
the elements of L are singular. Note that any singular element of L must be reducible 
as r U with r € [*T|, r smooth, because we have that any divisor numerically 
equivalent to T— f is not effective as deg£ < rk E. As h°(X, OxOFp)) = 1 , for all P € C, 
by Bertini's theorem all the elements of L are singular only if !fp is fixed and T varies in 
a subsystem of | T| of dimension deg E. This is impossible as h° {X, CMT)) = deg E. ■ 

Lemma 4.3. Let E be an ample vector bundle over an elliptic curve C such that 
deg E < rk E. LetX = P(E), and let D ~ aT+n*B be a line bundle on X, with b = deg B, 
b + (a- 1 > 0 and h°(X, Ox{Tj) > 3. Then the hypothesis of Lemma 4.1 are 

satisfied, with A = f T. 

Proof. If (a - 1 )/i - (£) + b > 0 then Lemma 2.4 gives h x (X, OxiD - A)) = 0. 
Moreover as each element of | T| is smooth, because we have that any divisor numerically 
equivalent to T — f is not effective as deg£ < rk£, the condition h°(X, Qv(T)) > 3 
shows that it is possible to find a smooth element 5 € |T| containing each fixed pair of 
points R,QeX. ■ 

The following lemma will be very useful to obtain the vanishing condition required 
by Lemma 4. 1 in many borderline cases. The notation used here is the classical notation 
used by Atiyah in [1]. 
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LEMMA 4.4. Let E be an indecomposable vector bundle over an elliptic curve C with 
rank E = r and deg E = d. Let X = P(E) and let n:X—*Cbe the natural projection. Let 
D = aT + **(B)for a line bundle B with deg 5 = b and letA = ? + tt*(Oc(P)) where P 
is a point in C. 

yt£=p&+b- J =0 , it is possible to choosePeC such that h\X,D- A) = 0. 

PROOF. It is enough to show that h 1 (C, S°- 1 E ® B ® O c (-P)) = 0. 

Since degS 0-1 /? <g> B ® Oc(—P) = 0 by Riemann Roch it is enough to show that 
A o^-i£ ^ 5 <g) O c (-P)) = 0. Because F'^E) is a direct summand of E^ a ~ l) it is 
enough to show that h°(E^ a - l) ® B <g> O c (-P)) = 0. 

Let h = gcoV, r). Then by [1] Lemma 2.4 and 2.6 it is 

(5) E = E? ®Ff, 

where d' = degtf ■ jf and / - rank£' = £ so that gcd^',/) = 1, F A is as in [1] 
Theorem 5 and £' is indecomposable. Being / and d! relatively prime, the condition 
+ b - 1 = 0 shows that / divides (a -I). Therefore following [8] Proposition 1 .4 
it follows that 

(6) £'® (fl - 1) = 0(F r( <8)L 1 ) 

Therefore putting (5) and (6) together we get 

£*- ! > = {E' <g> Fa)«*- !) = 0(F r , 8 £f) 0 ff-". 

Theorem 8 in [1] shows that tensor powers of F/ 's are direct sums of F*'s so we 
conclude that 

j 

It is then enough to show that for all / it is h°(F rj ®Lj®B® Od-P)) = 0. 

Let Lfj> = Lj®B® Oci-P). Recall that the F rj are obtained as successive extensions of 
each other by O c , i.e. for every r we have the sequence (4) (see proof of Proposition 2.11). 

This shows that it is h°(F rj <g> L)j>) = 0 unless Ljj> = Oc- It is then enough to choose 
a point P such that Lj®B® O c (-P) f O c for all / Since B is a fixed line bundle and j 
runs over a finite set, a P that works for all j can certainly be found. ■ 

The following theorem collects our results for the case rk E = 3. 

THEOREM 4.5. Let E be a rank 3 vector bundle on an elliptic curve C and let 
D = aT+bfbea line bundle onX= ?{E). 

(a) IfEis indecomposable then 

(al) ifd = 0 (mod 3), D is very ample if and only ifb + an~(E) > 3, 
(a2) ifd = 1 (mod 3), D is very ample ifb + ap,~{E) > I, 
(a3) ifd = 2 (mod 3), D is very ample ifb + ap.~{E) > \, 

(b) ifE is decomposable, then D is very ample if and only ifb + ap~(E) > 3 except 
when E = Ei@ E 2 , with rk£, = 1, rk£ 2 = 2, deg£ 2 odd and deg E { > In 
the latter case the condition is only sufficient. 
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Proof. Firstly we consider the case £ indecomposable and normalized. By 
Lemma 2.7 and Proposition 2.10 only the cases d = 1 and d = 2 need to be consid- 
ered. 

Case 1. d=\. 

Let A be as in Lemma 4.2 and notice that h°(A) = deg£ + 3. By Remark 2.6. We can 
assumefc+f = 1 +f, where e = l,2,3.If£ = 2,3then(a-l)/i _ (£)+&-l = 6+f-f > 0. 
If e = 1 Lemma 4.4 still allows the use of Lemma 4. 1 . Therefore Lemma 4. 1 and 4.2 can 
be applied. Let 5 be a smooth element in |T + f P \. It is enough to check when D\ s is 
very ample. By [6], Proposition 1.1,5= P(£') where £' is a rank 2 vector bundle on the 
curve C, with deg£' = 2, defined by the sequence 

(7) 0 -» O c (-P) -^E^E'^O. 

As Num(5) is generated by T and/, with T = T\ s and/ =f\ s , D\ s = aT' + bf and 
moreover by Section 3, D\s is very ample in our range if and only if b + an~(E') > 3. If 
E' is indecomposable then n~(E') = y.(E! ) = 1. If E' is decomposable, i.e. £' = //©G, 
both n(H) > //(£) = | and /x(G) > /*(£) = } because from (7) there exist non zero 
surjective morphisms E — * H and E — ► G. 

Hence deg// = degG = 1, n(H) = /i(G) = 1 and pr(E') = 1. In every case the 
condition b + a/j~(£ ') > 3 is satisfied in the range under consideration. 

Case 2. d = 2. 

Let /I and S be as in Lemma 4. 1 and 4.2. In this case it is h°(X, A) = 5 > deg£ + 3. 
By Remark 2.6 we can assume b + a\ = 1 + f with e = 1, 2, 3. If e = 3 the hypothesis 
of Lemma 4.2 are satisfied. If e = 2 Lemma 4.4 still allows the use of Lemma 4. 1 . 
Therefore it suffices to investigate the very ampleness of D\s. Let S = P(£') with E' a 
rank 2 vector bundle on C, with deg£' = 3 defined again by (7). If E' is indecomposable, 
D\ s = aT + bf is very ample if and only if b + aiT(E!) > 1, i.e. b + \a > 1 i.e. for all 
D in the range under consideration. 

If E' is decomposable then £' = //© G, with deg //and deg G > = f . Hence we 
can assume deg H = 1 and deg G = 2. By Lemma 2.8, n~(E) = 1 and the very ampleness 
condition is b + a > 3 which is satisfied by every D in the range under consideration. 

Notice that in the case b + \a = |, i.e. e = 1, a very ampleness result for all D in our 
range cannot be expected. For example, D ~ 2T is not very ample as D\y is not very 
ample for each smooth surface Y € |T| by Section 3. 

If £ is indecomposable but not normalized, the result is obtained by similar compu- 
tations (see (3)). 

To prove (b), let now £ be decomposable. Firstly we prove the sufficient condition. 
By Proposition 2.2 it suffices to prove that tt»(D) is very ample. Let us consider tt.(D) = 
^(E) ® OdB) = ® q A q where A q is an indecomposable vector bundle on C for all q. 
5"(£) <g> OdB) is very ample if and only if A q is very ample for all q i.e. if n(A q ) > 3 for 
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all q. This condition is satisfied if mity n(A q ) = ^'(^(E) ® O c (fi)) = b + a/x _ (£) > 3 
which is what we wanted to show. 

To prove the necessary condition, two cases will be considered: 

i) £ is sum of three line bundles, E = W<$>G®H respectively of degrees w <g<h. 
By Lemma 2.8 /T(£) = w, and d = deg£ = w + g + h. From [6], Proposition 1.1 it 
follows that P(G 0 H) is a subvariety of X corresponding to a line bundle numerically 
equivalent to T - wf while P(W) is an elliptic curve T on X, isomorphic to C, which is 
numerically equivalent to T 2 +xTf, for some x G Z. As the cycles corresponding to P( W) 
and P(G ®H)do not intersect, from {T 2 + xTf)(T - wf) = 0 we get x = -(g + /»). If D is 
a very ample line bundle on X, D\r is very ample, hence £>r = b + ayT{E) > 3. 

ii) £ = //0 G where rk// = 1, rkG = 2, h = deg//, g = degG. As in i), we get 
that Z = P(H) is numerically equivalent to f 1 — gTf and the very ampleness of D\z 
implies b + ah >3. IfA<f this concludes the proof. Otherwise let us denote by Y the 
smooth surface P(G). As usual Num(y) is generated by V = T\y andf =f\y- The very 
ampleness of D implies the one of D\ y and by Section 3, D\ y is very ample if and only if 

b + flf > 3 if g is even and, 

b + a* > 1 ifgisodd. 
If g is even, a necessary condition for the very ampleness ofDis6 + a§ > 3 i.e. 
b + an~(E) > 3 which is the desired condition. 

If g is odd, necessary conditions for the very ampleness of D are both b + ah > 3 
and b + a\ > 1 . Hence only the sufficient condition b + a/i"(£) > 3 is obtained in this 
case. ■ 

5. Rank r. To deal with the case of E vector bundle on an elliptic curve C, of rank 
r > 3 we need to recall first the following 

THEOREM 5.1 ([8]). Let E be a vector bundle of rankr on an elliptic curve C. E is 
ample if and only if every indecomposable direct summand E t of E has deg £, > 0. 

Our method based on Lemma 4. 1 , 4.2 and 4.3 forces us to investigate first the case 
deg E = 3, then deg E = 1 , 2 and finally deg£ > 4. 

5.1. deg£ = 3. 

THEOREM 5.2. Let E be a vector bundle over an elliptic curve C, with degE = 3 
and rank E = 4, and let D = aT+ bf be a line bundle onX= P(£). Then the following 
conditions hold: 

i) IfEis indecomposable, and b + \a > J, D is very ample if and only ifb + a> 3. 

ii) IfE is decomposable and ample, andb+%>\,D is very ample ifb+%> 2, 

Proof, i) If deg £ ■ /j°(T) = 3 then we can apply Lemma 4.3 and 4. 1 with A = ( T, 
as b + (a - 1 )n~(E) = 6+|a-|>0by hypothesis. Hence D is very ample if and only 
if D\ s is very ample, where S is a suitable element of |T|. There exists a vector bundle 
E' with rk£" = 3, deg£' = 3, given by 

(8) 0-+ O c ->£— 
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such that S = P(£') and Num(S) is generated by T and /', where T = T\ s and/ =f\ s so 
that ^15 = 07" + */. 

By Section 4, D|s is very ample if and only if b+a > 3. Indeed if E' is indecomposable 
the necessary and sufficient condition for the very ampleness is b + a > 3. If £' = ©, E\ 
then /t(£') > | for all i. Hence the only possibilities for a decomposition of E' are: 

1) E' = F0 G 0 H, with F, G, H line bundles all of degree 1. 

2) E' = H®G, with rank G = 2, rank H = 1, degG = 2, deg//= 1. 

Again by Section 4 in both the above cases, the necessary and sufficient condition for 
the very ampleness of D\s is b + a > 3. 

ii) Let us suppose that E is decomposable and ample. Then the possible decomposi- 
tions for E give n~(E) = §, j, 5- Note that the condition b + | > | allows us to apply 
Lemma 4.3 with A = T in any case. If S is the usual element of |T|, we get that D is 
very ample on X if and only if D\s is. If S = P(£')> could be decomposable. In this 
case n~(E') > \. Therefore the condition b + f > 2 guarantees the very ampleness of 
D|s by Section 4. ■ 

Because a > 2, Theorem 5.2 immediately gives the following: 

COROLLARY 5.3. Let E be an ample vector bundle over an elliptic curve C, with 
deg£ = 3 and rank£ = 4, and let D = aT + bf be a line bundle on X = P(£) with 
6+f > \.Ifb+* > 2 then D is very ample. 

THEOREM 5.4. Let E be an ample vector bundle over an elliptic curve C, with 
deg£ = 3 and r = rank£ > 4, and let D = aT+bf be a line bundle on X = P(£) 
with b + ap,-(E) > |. Then D is very ample ifb+*>2andb + *>\. 

PROOF. Proceed by induction on r = rank£. If r = 4 the inductive hypothesis 
is verified by Corollary 5.3. Let r > 5. It is /i°(*,T) = h°(C,E) = 3. Notice that 
H~(E) < < \ < |. Therefore b+(a - > 0 and Lemma 4.3 can be applied, 

with A = T. Let S = P(£') be as in (8) with deg(£') = 3 and rk(£') = r - 1 > 4. Num(S) 
is generated by T = T\ s and /' =J] S , so that D| s = at + 6/. Notice that E' is ample 
being a quotient of E. Notice that /z"(£) < Indeed there exists a map from at 

least one direct summand E k of E and the summand Ej of £' which realizes and 
so we get f r(E) < n(E k ) < ii(Ej) = n'iE'). As b + a/x-(£') > ft + a/i~(£) > |, by 
induction D\ s is very ample ifft+|>2, ft+f>j. ■ 

5.2. deg£' = 2. 

THEOREM 5.5. Let E be an indecomposable vector bundle over an elliptic curve C, 
with deg£ = 2, rk£ = r > 4. Let D = aT + bf be a line bundle onX= P(E). Then the 
following conditions hold: 

i) Ifr = 4,D is very ample ifb+\> 2. 

U) Ifr>*>,Disveryampleifb+\>2,b+\>\andb + a\> \ + \. 
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Proof. Let A be as in Lemma 4.2 and notice that h°(A) = 2 + r > deg£ + 3. By 
Remark 2.6 and the assumptions in i) and ii) we can assume 

(9) b + a- = \ + -, 

r r 

where e = 1, 2 . . . ,r. If b + a\ > 1 + J the assumptions of Lemma 4.1 and 4.2 are 
satisfied. If 6 + = 1 + ^ the line bundle D-A has degree 0 and Lemma 4.4 shows that 
Lemma 4. 1 can be used if 

(10) b + a->l + -. 

r r 

Therefore condition (10) can be rewritten using (9) as 

(11) b + a->\ + -. 

r r 

Let S = P(£')> where E' is as in (7), where deg£* = 3, rk£* = r - 1 > 3, Num(S) is 
generated by 7" = T\ s and/ =f] s , and D\ s = aT+bf. Notice that E' is indecomposable 
or decomposable and ample because /i~(£') > n~(E) = \ > 0. 

To prove i) assume r = 4. In this case rk E' = 3 anddeg E' = 3. If E' is indecomposable, 
as b + an~(E') > b + an~(E) = b + f > 1, by Theorem 4.5 D| s is very ample if 
b + a > 3. By the same theorem if E' is decomposable and ample then D\s is very ample 
if b + an~(E') > 2. The possible values for pT{E') are 1 or \. Therefore by Theorem 4.5 
it follows that D\ s is very ample if 

(12) fe+|>2. 

Putting ( 1 1 ) and ( 1 2) together it follows that in the case r = 4 D is very ample if b + f > \ 
andZ> + f >2i.e. b + % > 2. 

To prove ii) assumer- 1 > 4. Since6 + a^"(£') > b+ap~(E) = Z>+a^ > l + I > 2 
from Theorem 5.4 it follows that D\ s is very ample if b + 1 > 2 and 6 + 1 > j . Therefore 
ifr>5,Disveryampleiffc + § > 2, 6 + f > ^and^ + a^ > 1 + 7. ■ 

THEOREM 5.6. Let Ebea decomposable and ample vector bundle of rank r > 4 over 
an elliptic curve C, with deg E = 2. Let D = aT+bf be a line bundle onX= P(£). Then 
the following conditions hold: 

i) Ifr = 4,D is very ample ifb+j>2andb + a^i~{E) > |. 

ii) Ifr>5,D is very ample ifb + a/j~(£) > 1 + \, b + f > } and b + f > 2. 

Proof. Let A be as in Lemma 4.2 and notice that h°(A) = r + 2>deg£ + 3. Also 
notice that fi~(E) < n(E) = \. In our hypothesis we have b + (a - l)n~(E) - 1 > 
b + an-(E)- \ - 1 >0. 

Therefore Lemma 4. 1 and 4.2 can be applied. Let S = P(E'), where E' is as in (7) 
where deg£' = 3 and rk(£') = r - 1 > 3. If r > 5 we can apply Theorem 5.4 to D\ s . 
Indeed b + an~(E') >b + an~(E) > 1 + \ > f . Hence £>| 5 is very ample if *+ } > } 
and&+§>2. 
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If r - 4 we can apply Theorem 4.5. If E' is indecomposable then D\s is very ample if 
b+a > 3. If£' is decomposable, noticing that £' is ample, the condition is b+an~(E') > 2. 
In this case fi~(E') can be 1 or \ and hence the worst sufficient condition becomes 
ft+§>2. ■ 

Theorem 5.5 and 5.6 give the following corollary. 

COROLLARY 5.7. Let E be an ample vector bundle over an elliptic curve C, with 
deg£ = 2 and let D = aT+bf be a line bundle on X = P(£). Then the following 
conditions hold: 

i) IfnnkE> 5, then D is very ample ifb +an~(E)> 1 + J, fc+f >2andb+* > ±. 

ii) If rank E = 4,D is very ample ifb + ap.~{E) > § and b+j>2. 

5.3. deg£ = 1. Note that if deg£ = 1, E is ample if and only if it is indecomposable. 

THEOREM 5.8. Let E be an indecomposable vector bundle over an elliptic curve C, 
with deg£ = 1, rk£ = r > 4. and let D = aT + bf be a line bundle on X = P(£) with 
b+j>l. Then the following conditions hold: 

i) If r = 4, D is very ample ifb + § > 2. 

ii) Ifr = 5,Dis very ample if b+ § > \ andb+ f > 2. 

Hi) Ifr>6,Dis very ample ifb+ 7 £ } >l + T ^andb+j>2. 

PROOF. In our hypothesis it is (a - 1)^ + b - 1 > 0. Lemma 4.4 allows us to apply 
Lemma 4.1 and 4.2 when (a - 1)J + * - 1 > 0, noticing that if A = T + <f P it is 
h\A) = 1 +r > 5. If S = P(E% where E' is as in (7), notice that E' is ample because^ is. 
It is rk£' = r - 1 > 3 and deg£' = 2. Let us now distinguish the three cases according 
to the values of r. Theorem 5.4 and Corollary 5.7 will be used. 

i) If rank £ = 4, rank £' = 3, and D\ s is very ample if b + fa > | when £' is 
indecomposable while if £' is decomposable and ample the condition is b+ap.~(E') > 2. 
In the worst case //"(£') = ^ so we have b+%>2. 

ii) If rank £ = 5, rank £' = 4, and D\ s is very ample if b+ati~(E') > \, and b+ f > 2. 
As £' is indecomposable or decomposable and ample, in the worst case n~(E') = j and 
so it is enough to ask b + f > \ and b + | > 2. 

iii) If rank £ > 6 then rank £' > 5, hence by Corollary 5.7 D\ s is very ample if 
b+an~(E') > l+jcf, *>+§ > }andfe+f > 2.If£'isindecomposablethen/i~(£')= 
while if £' is decomposable and ample, then /x~(£') = min(j, j^^) with s = 1 • • ■ r — 2. 
So the condition b + apT{E') > 1 + ^ can be substituted by b + ^ > 1 + ^ which 
implies the condition ft + f > 5. ■ 

5.4. deg£>4. 

THEOREM 5.9. Let E be an ample vector bundle over an elliptic curve C, with 
deg£ = d > 4, rk£ = r > 4 and d < r. Let D = aT + bf be a line bundle on 
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X = ?{E). Ifb + > 2 and b + (a - 1 )fi~(E) > 0 f/ie/i D is verv ampfc. 

Proof. The proof proceeds by induction on r. If r = 4 the smallest possible value of 
//-(£) is i. Since d > 4 it is ^ < i. Therefore 6 + >/>+f>6+^T>2. 
Lemma 2. 1 1 and Theorem 2.5 conclude the proof of the initial inductive step. Let us 
suppose the statement true for r - 1 and prove it for r. Let A be as in Lemma 4.3 and 
notice that h°(A) = d>4. Therefore Lemma 4.3 and 4. 1 can be applied. By considering 
S = P(£') where E' is as in (8), we get that E' is ample with deg£' = d > 4, rk E' = r 1 = 
r - 1 > 4. Because n~(E f ) > n~(E) it is b + (a - 1 )fi~(E') >b + (a-\ )n~(E) > 0 and 
again b+j^>2. Hence we can conclude by induction hypothesis. ■ 

Remark 5.10. Note that in the above theorem, if E is indecomposable, by normalizing 
E we can always assume d < r. 

In a very particular case we can say a little more: 

PROPOSITION 5.11. Let E be an indecomposable vector bundle over an elliptic curve 
C, with degE = d>4,rkE = d+\,andletD = aT + bfbea line bundle onX= P(£). 
lfb + (a-\)£j>0andb + a>2thenD is very ample. 

PROOF. It is h°(X,T) = d > 4. Our hypothesis b + (a - t)^ > 0 shows that 
Lemma 4. 1 and 4.3 can be applied. 

By considering S = P(£')> where E' is as in (8) it is deg E' = rk E' = d. A sufficient con- 
dition for the very ampleness of D\s is b+an~(E') > 2 by Lemma 2. 1 1 and Theorem 2.5. 
We claim that in this case n~{E') = 1 . If E' is indecomposable it is n~(E') = fi(E') = 1 . If 
E' is decomposable we can suppose that E' = ®Gj, with r, = rk Gj > \,dj = deg Gy > 1 
(as n(Gj) > n(E) = jjjj) and £</, = E/> = </• Hence we have J > for all y, which 

implies r 7 — ^ < ^ < I , for all y. Hence r, — fi^ < 0, for all j i.e. d } = r, + 5 y with s y > 0, 
for all / But d = Y.dj = £(</, +*,) = </ + E and = 0 j. e . j, = 0, for all / Hence 
»-(E')=l. m 
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NEW ALMOST PERIODIC TYPE FUNCTIONS AND 
SOLUTIONS OF DIFFERENTIAL EQUATIONS 

BOLIS BASIT AND CHUANYI ZHANG 



Abstract. Let A" be a Banach space and J G {R + , R}. Let n and rio be two sub- 
spaces of C(J, X), the Banach space of bounded continuous functions from J to X. We 
seek conditions under which n + Flo is closed in C(J, R). This led to introduce a gen- 
eral !A\ I , i J. A .) space, which contains many classes of almost periodic type functions as 
subspaces. We prove some recent results on indefinite integral for the elements of these 
classes. We apply certain results on harmonic analysis to investigate solutions of dif- 
ferential equations. As an application we study specific !A\ I r (J, R) spaces: the spaces 
of asymptotic and pseudo almost automorphic functions and their solutions of some 
ordinary quasi-linear and a non-linear parabolic partial differential equations. 

1 . Notation, Definitions and Introduction. Throughout this paper, X denotes a 
Banach space and X* denotes the dual space of X. Let J e {R + , R}. Then £7(J, X) will 
stand for the Banach space of bounded continuous functions <p from J to A 1 with norm 
||<^|| = sup^j || </3»(xr)|| and C U (J,X) will denote the subspace of C(J, X) consisting of the 
uniformly continuous functions. If (v?„), </> C C(^X), we write <p„ —* <p if and only if 
II V* - </> II — ♦ 0 as n — ► oo. In case that X = C, we will omit X from our notation and write, 
say, C(J) for C(J, X). The translate and the difference of tp by 5 6 J are respectively the 
functions R s <p(i) := <p(t + s) for all / e J and A s <p := R s <p - (p. Let J be a subspace 
of C(J,X). Then f is said to be translation invariant if R s ? c 7 for all s G J. If 
<p e C(R, X) then L(ip) will stand for the subspace of C(K X) generated by all translates 
of<p. We denote by 

(1.1) <PmV,X) = We a\X) : Mm -1- jf ||^)|| ds = 0}, 
and 

(1.2) KJ,*) = W e CUX) : \\m-l-J^ is + x )d s = MM}, 

where c = 0 if J = R + and c = -t if J = R, and the limit in (1.2) exists uniformly 
with respect to jc 6 J. The set TJ%Po(J,X) is a translation invariant, closed subspace of 
C{ J, X) which does not contain any constant functions except 0. J, X) is introduced 
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by Zhang [26-28] and its elements are called ergodic perturbations. We remark that the 
elements of (PJ%Po(J,X) are not necessary uniformly continuous. £(J,^0 is called the 
space of ergodic functions [2, 13, 18, 22] and M(tp) is called the mean of ip. 

Let (f G C(J,X). The function <p is called (weakly) almost periodic if the set {R s ip : 
s G J} is (weakly) relatively compact in C(J,X). Denote by ( ( H>ffl'(J,Xj) Jl%P(J,X) 
(J = R + ) all such functions. If J = R, then the space of almost periodic functions will 
be denoted by J%P(R,X) (see [1], [1 1], [17]). Recently, Ruess and Summers [22, Propo- 
sition 2.1, Theorem 2.4] and Basit [2, Theorem 2.3.4 and 2.4.7] proved that 

(1.3) t W^P(J,X) C £(J,*). 

We recall that a subset E of R is called relatively dense in R if there exists a finite 
subset {t\ , t 2 , ■• -,t m } C R such that R = U£Li('« This implies that there exists / > 0 
such that ]x, x + /[ D E ^ 0 for all x G R. A function <p G Cu(R,X) is said to be recurrent 
if, for each e > 0 and N > 0 the set 

E(€ 9 N, if) = {r G R : + r) - <M0ll < e, M < 

is a relatively dense subset in R. The set of all the recurrent functions will be denoted by 
r<M). 

It is known that r(R,X) is not linear [14, 17]. Nevertheless, J%P(R,X) is a linear class 
of r(R,*) and also, L((p) is a linear class of r(R,X) for all <p G r(R,X). 

A function G G(R,*) is said to be almost automorphic if it is recurrent and for each 
c > 0 and N > 0 there exist 6 > 0 and M > 0 and a relatively dense subset 5(5, A/, </>) 
such that 

5(5, M, <p) - B(5, M, tp) C E(c, N, ip). 

The set of all almost automorphic functions will be denoted by JW(R,X). 
The following needed in the sequel statement is obvious. 

PROPOSITION 1.1. Lettp G AA(R,X) and xp G CuiKX)- Let for each e > 0 an d 
N>0 there exist 6>0andM>0 such that 

E(6 t M,<p)cE(e,N,xP). 

Then tp G JW(R, X). 

If the range of ip is precompact, the above definition is equivalent to the definition 
which S. Bochner [10] introduced for numerical almost automorphic functions. The in- 
vestigation of Veech [24] and Basit [6] (see also references in [2]) showed the relation- 
ship between almost automorphic functions and almost periodic functions in the sense of 
Levitan. The extension to vector-valued functions and many equivalent definitions are 
obtained in [6, 19,24,25]. 

We shall denote by WWP 0 (J,X) = {tp G W&P(J,X) : A/(||^( )||) = 0}, and 
G>aX) = {tp G C(A,X) : lim,,,.^ </>(/) = 0}. It is obvious that Co(J,X) C 
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Let 

(1.4) n 0 (J,*)€ {<PJW 0 (J,X),G(J,X),Wm(XX)}. 
U C (R,X) denotes any class of functions satisfying: 

(1.5) n c (R, X) is a translation invariant, closed subspace of C U (R, X) containing all the 
constant functions. 

(1.6) The map m: T\ C (R,X) n f (R + ,A^ defined by m(<p) = <p\ R * is an isometry. 

(1.7) n c (R,A0 is closed under multiplication by characters. 
n r (R,^) denotes any class of functions satisfying (1.5), (1.7) and 

(1.8) T\ r (R,X) C r(R,X). 

It follows from [2, Proposition 2.1.7] that n r (R,X) satisfies ( 1 .6) and hence 

(1.9) u r (R,x) c n r (R,*)andn r (R\;o c n c (R\x). 

Define 

(1 10) *n c (J,*) = n c (j, X) + n 0 (J,*), 

We have 

(i.ii) xn r (j,x) c mc(j,x). 

If tp = V> + £ with %l) e U C (J,X) and £ G n 0 (J,*), then xp will be called the almost 
periodic type part and £ the ergodic perturbation. We write 

p 0 ^n c (j,^o, c 0 ati c (j,x), w 0 an c (j,x) 

instead ofJm c (J,X)\f 

n 0 (j,*) = <pm{±x), G(ix), <wmv,x) 

respectively. 

The space -#n r (J, X) is a generalization of some spaces of almost periodic type func- 
tions. Here, we mention some of them: 

JUtP(J,X), the space of asymptotically almost periodic functions, WJ%P(J,X), the 
space of weakly almost periodic functions (see [2, 13, 15, 19, 22]); (PJ%P(J,X), the space 
of pseudo almost periodic functions (see [26-28]); A%J%(J,X), the space of asymptot- 
ically almost automorphic functions (see [2]). For other classes of functions satisfying 
(1.5H18) see [2]. It follows easily that U ( (R,X) are A-classes introduced in [2, Defini- 
tion 2.2.1]. We prove that AY\ r (J, X) n C U (^X) is a A-class. 

In this paper, we mainly study the space &n r (J,X). We prove that these classes are 
closed subspaces of <T(J, X) in section 2. Some recent results on indefinite integral ob- 
tained in [2] and [27] are extended in section 3. In section 4, we study the harmonic 
analysis introduced in [2] for these classes and give some of its application to ordinary 
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differential equations. Since the elements of these classes are not necessary uniformly 
continuous, the application of the results of [2] is not direct. In Sections 5, we will inves- 
tigate a specific An r (J,X) space: pseudo and asymptotically almost automorphic func- 
tions and their solutions of quasilinear ordinary differential equations. We study the so- 
lutions of non-linear parabolic partial differential equations in section 6. 

2. Properties of the Spaces Ml c (J,X) and Ml r (J,X). In this section we study 
Y\ 0 (J,X), AI\ C (J,X) and AI\ r (J,X). We prove that every AI\ r (J,X) class is a closed 
subspace of C(J,X) and every C 0 .#n c (J, X) class is a closed subspace of C U (J, X). 

PROPOSITION 2.1. Ifip€ -fllTcM,*) (-3rir(J,*)) and \b G C(R,X) such that t/;|j = 
<p, then 

(i) R s rPU G WlMX)(Wl r (J,X)) for alls G R, 

(U) if, e Jm c (j,x) n G(J,*) (iUWJ, A) n (Mxj)for aiif e z,'(R). 

Proof. First, we prove the case </? G Tlo(J,X). 

(i) If J = R then ^ = <p and (i) follows from the fact that Ylo(R,X) is translation 
invariant subspace of C(R,X). If J = R + , one can choose tpo G l~(R, X) such that t/>o(0 = 
<p(t) for / > 0,Vo(0 = f(0)(t+ 1) for -1 < t < 0, and i/; 0 (0 = 0 for / < -1. A direct 
verification shows that #,i/>o|j € n 0 (J, X) for all s G R. Since R s (\p - xpo)\j has compact 
support for all s G R, we conclude that R S W - Vo)|j G n 0 (J,A). This implies that 

e n 0 (J,^). 

(ii) If h > 0 and/ = Xfft*] me characteristic function of [0,h], then V */(0 = 
So i)(t + x)<ic. Direct computation for each of the three cases of ITo(J, X) shows that 

V *Ai € n 0 (J, X) n COW If * < 0, then $1$ +x)dx = +* + h)dx m 
- So h Rh^K' + x)dx. By (i), we have that R h ib\j G n 0 (J, X). Since —h > 0, it follows that 

V R h xp(t +x)dx\j belongs to Ho(J,X)C\ C U (A,X). For </ > c, define a function g Cjd on R 
such that g c4 {t) = 1 for / G [c, </] and g C)</ (/) = 0 otherwise. We have that 

0 * fivKf ) = £w+x)dx-£w+x)dx (I € R). 

Hence, we get i/;*g Ci</ |j G rio(J,AyX^(J,A)forall c,*/G R. If//(0 = a*gc t ^(0isa 
step function, then, by the linearity of rio(J, X) we get//*i/>|j G Ilo(J,A^nCL(J,^0. Since 
/ G £'(RX there exists a sequence of step functions {H„} such that ||// n — f\\ L \ — ► 0 as 
n — ► oo. This implies that \\ip*H„—xp*f\\ — ► Oasn — • oo. Hence ||i/;*// n |j—V;*/lj|| — » 0 
as n — » co. Therefore, tb G n 0 (J, X) D G(J, A). 

Secondly, let </? G ^U1 C (J,X), <p = C|j + £, where < G n c (R,A) and £ G n 0 (J, A). 
The case ip G J4TI r (J,A) can be proved exactly in the same way. Set t/ = fb — £. Then 
T7|j = ^ G rio(J,A). By the first part (i), weget/?,r/|j G n 0 (J,A) and by property (1.5) of 
n c (R, X), we have fl,C G U C (R,X). This implies that ^| j = +R s riU <= ^n c (J, X). 
By the above n G n 0 (J, JQn G(J, A) and from the inclusion C*/ G L«) C n c (R, A) 
for all / G L\R) by [2, Lemma 1.2.1], the proof of (ii) is complete. 
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Corollary 2.2. Let ip G ATl c (J,X)(jm r (J,X)) and Pip(t) = So<p(x)dx. Then 

a*/v g W,J0naJ^(w^nC K (J^)/ O r fl ///> e J. 

Proposition 2.3. C 0 An c (J,X) (C 0 An r (J,X)) is a closed subspace of C U (XX). 
Moreover, C 0 Xn c {XX) = Y\ C (S,X) 0 G(J,*) (C 0 AT\ r (J,X) = U r (J,X) 0 Co(±X)). 

Proof. It is sufficient to prove the case ip G C Q An c (J,X). Let = xp\j + £, where 
V> G n c (R, A) and £ G a(J,*)- We show that 

(2- 1) IMI < IMI- 

Indeed, ||?|| = + *|| > + RAW > ~ M for s G J. Taking the 

limit when s — ♦ oo and using (1.6) and the fact lim,_^o \\Rs£\\ = 0, we get (2.1). 

Now, let {<?„} be Cauchy in C 0 Jin c (J,X). Then <p„ = xp„\j+^„, where fjj n G n c (R, A) 
and G Co(XX). It follows from (2.1) that {V>„} is Cauchy too. By property (1.5), 
we conclude that {V>„} converges to ip G Yl c (R,X). This implies that {£„} is a Cauchy 
sequence in Co(R,X) and hence converges to £ 6 Co(J,X). Hence, {<p„} converges to 
-01 j € CoXndJyX). Therefore, CoAU c (J,X) is a closed subspace of Cu(J,X). 

Finally, we show that C 0 An c (J,X) = Yl c (J,X) 0 C6(J,X). Assuming that for <p from 
C 0 ^n c (J,^), there are \pi G n c (R, *) and £, G Co(<S,X), i - 1,2, such that <p = ft|j+£/- 
Then 0 = (tf, - V> 2 )|j + (£■ - 6) By (2.1), - V2II < 0- Therefore, 0 ( = V2 and 
£1 = £ 2 - The proof is complete 

We do not know whether or not P 0 M\ C (J,X) and fV 0 SM e (J,X) are closed subspaces 
ofC(J,X). 

PROPOSITION 2.4. JWl r (J,X) « a closed subspace of C(J,A> Moreover, 

jm r (j,x) = n r (j, A) 0 n 0 (J,A). 

Proof. By Proposition 2.3 the statement holds for the case that rio(J, X) = Co(J, X). 
Now, let rio(J,*) = <PSW 0 (t,X) y let tp = V|j + where V € n r (R,*) and £ € 
¥WP 0 (J,X). We show that 



(2.2) V>(R)C*>(J), 

where V(R) denotes the range of xp and y?(J) denotes the closure in A' of the range of tp. 
Assume that (2.2) is false. This implies there exists to G R such that xp(t 0 ) & <p(J). 
Since ip is recurrent, we can assume that / 0 G J. There exists an e > 0 such that 
inf jG j \\xp(to) - <p(s)\\ > 2c. Since 0 is continuous, there exists 8 > 0 such that 

(2. 3) inf||*<,iK') - R,Ms)\\ > « (I'l < 

Since /?, 0 t/> is also recurrent, for e > 0 and 6 > 0 there exists l (/2 such that each interval 
(x,x + / (/2 ) contains a number r with the property 



(2.4) 



KW+t)-jw</)||<! (W<«). 
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Let / and t satisfy (2.4) and let / + r G J. It follows from (2.3) and (2.4) that 

K*(/ + T)|| = K?(* + T)-^W + T)|| 

> I!***** +t) - R h m\\ - \\Rt 0 m - mc+t)| 

> inf||/^(/ + r)-*,>(0|| - sup ||/?, o i/>(0 - W + r)|| 

T T 

Since each interval (x,x+l ( / 2 ) contains a number r, it follows from inequality above that 

Km — f'U(s)\\ds>^-. 

t^oot-cJc " SV ~ 2/ </2 

This contradicts the fact that £ 6 fR3fP 0 (J,*). We have proved (2.2). It follows that 

IMI < IMI- 

Now, we can proceed exactly as in Proposition 2.3 to show that flTl r (J,X) is a closed 
subspace of C(J,X) and .3n r (J,*) = n r (J,X) 0 ££2fc(J, JQ. 

To show the assertion for the case that U 0 (J,X) = WJ%Po(J,X), let tp = ^Jj + £, 
where V € n r (R,*) and £ G W^P 0 (J,X). We need only to show that ||V>|| < IMI- For 
x* EX*, the composition function** o £ is in < W#tP 0 (J). Since 'M^tPo(J) C TWP 0 (J) 
[9, 4.3. 13], x* o f e (PJ%Po(S)- Note that the numerical function jc* o ip is recurrent and 
x* o p = x* o \p + jr* o £, it follows that 

I** o V>|oo < P o < I^HI^loo (x* G **). 

Therefore, < \\<p\\. The proof is complete. 

We remark that the above proposition gives a new proof that WJ^P(J,X) = 
WP(J,X) 0 <H>A%(J,X) [15]; and JUVP(J,X) = j*P(J,*) e a(J,*) [18, 19]. As a 
consequence, we have 

Corollary 2.5. The spaces ^n r (J, *) n G(J,*), C 0 J?Tlc(J,AO fl« A-classes. 

Now, we prove 

THEOREM 2.6. Let ip G G(R,^). 77ie/i V>|j G JOT r (J,A}(C 0 ^n c (J,*)) //am/ on/y 

</> e ^n r (j,^)(c 0 ^n c (j,^)) /or all f e l\R). 

Proof. Necessity follows from Proposition 2. 1 . We show sufficiency. Let h„{t) = 
n/2 for |/| < \/n and h n (t) = 0 otherwise. Since V G <T U (R,*), we conclude that 
A„*V(0 = W+x)dx -» - ooin C(R,*). Hence ||W|j- VIjII - 

0 as /i — » oo. By assumption, we have h„ * V|j G J?TI r (J,^),n G N. By Proposi- 
tion 2.4, m\ r (J,X)(C 0 m\ c (J,X)) is closed and hence the limit linv^o h n = 6 

j^n r (j,^)(c 0 ^n c (j,^)). 
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3. Indefinite Integral of Functions in ^n r (J, X). In this section we study the in- 
definite integral of functions in J%n r (J,X) (CoJ^n c (J,X)). Throughout the section, if 
<p € C(J,X) then P<p will denote the definite integral P<p(t) = # i ipix)dx. We extend 
some results obtained in [2, 7, 16, 27]. 

Theorem 3.1. Let tp G n 0 (J,*). Then Pip G Jm r (J,X)(C 0 AY\ c (J,X)) if and only 
if there exists a constant k G X such that Pip - k G n 0 (J, X)(C 0 JVI C (J, A")) . 

PROOF. Wc need only to prove the necessity. Let Pip G ATl r (J,X). Then P<p = 
0| j + where 0 G n r (R, X) and t G n 0 (J,*). We have 

A h Pip(t) = Pip(t + h)- P<p(t) = £<pit + x)dx 

and 

A h PiPit) = A„mu + = m +h)- mh + kc + - 

By Corollary 2.2, we get A h Pip G rio(J,*) for all A 6 J. Since n r (R,*) and n 0 (J,*) 
are translation invariant, A*V|j € T\ r iJ,X) and A/,£ G n 0 (J,A). By the uniqueness of the 
decomposition of elements of XI\ r iA,X), we get A*V> = 0 and A*£(/) = jj <^(/ + 
Since xb is continuous, we conclude that xp — k G A*. 
The case C 0 .fln c (J, can be proved in the same way. 

We remark that Theorem 3.1 gives a new proof of [27, Theorem 7] first proved by 
Zhang for the case that n r (R,*) = WPiR,X) and n 0 (J, X) = ¥WP 0 iJ,X). As a conse- 
quence of Theorem 3. 1 , we have 

Corollary 3.2. Let ip = VI j + £ G M\ r iJ,X), where \p G n r (R,A0 and £ G 

n 0 (J,^). Then Pip € -3n r (J,*) if and only i//>V|j € n r (J,*) and there exists k E X 
such that Pi ~ *Gn 0 (J,A> 

PROOF. We need only to prove the necessity. Let Pip = flj + n, where C G n r (R,A) 
and t/ G n 0 (J,A> Then, for A G J, 

A^(/) = A A Clj(0 + A A r,(0 

= A /I />V| J (0 + A A PC(/) 

Similarly, we have that A A <] j, jj t/>(/ + x)<£c G n r (J,A) and A/,n, jj £(/ + Jt)t£t G 
rio(J, A^. By the uniqueness of the decomposition, we get A/,(r/ — P£) = 0. This implies 
that there exists k G X such that P£ = t/ + k. Hence f% = <| j - k and /»V|j G U r iJ,X). 
Corollary 3.2 holds also for the case ip G CoflndJ, X). 

Now, we study sufficient conditions for P<p in JZTl r iS,X). We have 

THEOREM 3.3. Let ip G m r i±X)(C Q %n c i*,X)) and Pip G £(J,A> Then Pip G 

^n r (j,^)(c 0 ^n c (j,^)). 

PROOF. Lety> = V+£withV G n r (J,*)and£ G no(J,A). By Corollary 2.2, A h P(, G 
n 0 (J,*) n C(J, A) and by [2, Lemma 1.2.1], A h Pxp G L(<?) C n r (R,*) for all /i G R. 



NEW ALMOST PERIODIC TYPE FUNCTIONS 1 1 45 

This implies that A h P<p G AU r (±X) D C(J,*)- By Corollary 2.5 J?n r ( J, X) D C U (R,X) 
isaA-class. Since P<p G *E(R,X),P<p G AIl r (J,X)nCu(J,X) by [2, Theorem 3.1.2]. By 
Corollary 2.5, C 0 Sffi c (J,X) is a A-class, hence the statement follows directly from [2, 
Theorem 3.1.1]. 

4. Harmonic Analysis of Ml r (J,X) and Solutions of Linear Differential Equa- 
tions. In this section we introduce the spectrum sp^ n (<^) of the function <p G C(R,X) 
with respective to An r (J,X)(CoMl c (J,Xj) . This kind of spectrum proved to be a useful 
tool to examine solutions of differential equations of many function classes [2, 5, 8, 17 
Ch.6]. See also the recent works [4], [20], [23]. We apply the results on spectrum to the 
differences of functions and linear differential equations on the half line. Since the proofs 
for the case C 0 An c (J,X) are the same as in the case CoMl r {^X) we restrict ourselves 
to Sm r (J,X). 

Let if G C(R,X). Denote by lsmM>) = if e L '< R ) : /* 6 -3n r (J,*)}. Since 
fln r (J,X) is a closed subspaceof C{A,X),I^ Ur {ip) is a closed ideal of Z,'(R). We set 

(4- 1) vmjtv) = hull IsmM = {A G R :/(A) = 0,/ G ImMl 

We denote by sp(^) the Beurling spectrum of <p [21, pi 38]. Similar to the corresponding 
properties of Beurling spectrum, the following can be proved exactly as in [2]. 

Proposition 4.1. Let<p g C(R, X) andf e L 1 (R). Then 

(a) sp*n r (Y>) c S P(V)/ 

(b) sPan,(V*f) C sp^(v)nsupp/;/ G l'(R) such thatf G I A n r (<P) andf(X) ^ 0. 

(c) Let <p G G(KX). Then <p\t G Wl r (J,X) if and only //sp m (<^) = 0. 

THEOREM 4.2. Let ip G Cu(R,X) and let sp(<^) be separated from zero. 

(a) 7/A^|j = (R h <p - ip)\j e M\ r (J,X)forallh G R, then G m r {S,X). 

(b) lf<p\ s e m\ r (J,X), then P<pU G Ml r (J,X) D Q(J,X). 

PROOF. Corollary 2.5, Ml r (J,X) n C(J, X) is a A-class. Therefore (a) follows di- 
rectly from [2, Theorem 4.2.4]. 

(b) By [3, Corollary 4.4] Pip is a bounded uniformly continuous function and there 
exists k G X such that sp(/V + k) is isolated from zero. By Corollary 2.2, & h Pip\j G 
m r (XX) for all h£R. Therefore, P<p|j G AT\ r (J,X) by (a). 

Now we study the linear differential equation 

(4. 2) / + a,/ + aQ y = if (ip G #n r (J, *)) . 

We give sufficient conditions under which the bounded solutions of (4.2) is in JZU r (J, X). 

THEOREM 4.3. Ifa 0 , a\ G C in (4.2) are such that (iaf + iaa\ +a 0 ^0 for alia € R 
and ify is a bounded solution of (4.2), theny G %£\ r (S,X). 

PROOF. If J = R + , we define Y(t) = y{t) for / > 0 and Y(t) = y(0) cos( 1 - cos /) + 
/(0) sin / +y'(0)(sin 2 1)/2 for f < 0. It is easy to verify that Y is a solution of the differ- 
ential equation: 

(4.3) r+a i y + a 0 Y=tl> 
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where \p(t) = <p(i) for / > 0 and tp(t) = f + a\ Y' + a Q Y for / < 0. Since Y is bounded, 
by Landau-Esclangon lemma we conclude that Y 1 , Y" are also bounded (see [17, Propo- 
sition 4, p.95]). This implies that Y is uniformly continuous. It is easy to check that 
\j) € C(R,X). We prove that spj,,, (Y) = 0. Indeed, let a G R, there exists / G S(R) such 
that/(a) ^ 0. Wehwef*Y f '+f*a l Y'+f*a 0 Y=f*f. Hence Y*[f" +ayf + a<f\ = tfj*f. 
We have if" + a\f' + ao/)(or) = [(iaf + iaa\ + ao]f(a). From the assumptions we con- 
clude that the function g := f + a\f + aof satisfies g(a) ^ 0. From (4.1), it follows 
that a g sp^ n (y). Since a is arbitrary, we get sp^ n (K) = 0. By Proposition 4.1 (c), 
y=Y\jeM\ r r (J,X). 

In the same way, we have 

COROLLARY 4.4. Let <p G ATl r (J,X). Let A = u+iv with u^O. Then every bounded 
solution of the equation y' = \y + <p belongs to J%Tl r (J, X). 

THEOREM 4.5. //A = u + /v€C with u>0andip G JWl r (J,X), then the solutions 
of the equation 

dy 

(4.4) & = Xy + if 

of the form y(x) = e**[c + J^VC)^] are bounded on R + Furthermore, if J = R + , 
then y G An r (R*,X) and \\y\\ < m~'|M|. If J = R, then (4.4) has a unique bounded 
solution defined on R if and only if y(0) = - So° e~ x, (p(t)dt. In this case the function 
y 0 (x) = - JT 3 e**-*<p(t)dt is a bounded solution from M\ r (R,X). 

Proof. Direct verification shows that 

(4. 5) y{x) = e**[c + £ e~*V(0<tfJ, 

is a bounded solution of (4.4). If J = R + , then y G Ml r (R+,X) by Corollary 4.4. 
Since u > O^e^l = e"* — + oo as jc — > oo. For y to be bounded on R, we must 
have c + Jo e~ Xl <p(t)dt — ► 0 as x — * oo. This means that we must take in (4.5) 
c = — So° e~ Xl ip(t)dt. We note that the improper integral in (4.5) is convergent since 
\e~ x '<p(t)\ < |M|e" M ' for / > 0. Thus, the unique bounded solution of equation (4.4) 

canonlybe>-o(x) = -Xr^ (JC_, V(0^andwedohave|LvoWll < M^iT*"**" 
\\<p\\/u, so that^o is bounded. By Corollary 4.4, y 0 G AT\ r (R,X). 

When u < 0, we have thatjv 0 (x) = fi^ e*'- x) <p(t) dt is the unique bounded solution 

of (4.3) and^o G AT\ r (R,X). At the same time we have the similar estimate \\yo(x)\\ < 

IMI/M. 

Theorem 4.6. Let xp G C(R,X) and V>|j G AT\ r (J,X). Let <p(t) = e~ ivt ^{t) with 
v G R and let the Beurling spectrum of ip be separated from zero. Then all solutions of 
equation (4.3) belong to W\ r (J,X) n G(J,*). 

Proof. (4.4) has general solution y(x) = ^[c + So e~ ivt %p(t)di] y where c is an arbi- 
trary element ofX. By Theorem 4.2 (b), we conclude # <T'' w V(0*]|j is in J?n r ( J, X) D 
C(J, X). Since JWl r (XX) contains all constant functions and invariant under multiplica- 
tion by characters, y G J?n r (J,*) D C U (J,X). 
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5. Asymptotically and Pseudo Almost Automorphic Functions. In this section 
we will concentrate two new, specific ^Hl r (J,X) spaces. That is the space of asymptoti- 
cally almost automorphic introduced in [2] and the space of pseudo almost automorphic 
functions defined below. 

Definition 5.1. A function 0 G C(J,X) is called pseudo (asymptotically) almost 
automorphic if xp = <p\j + £, where if G JZA(R,X) and £ G <PWPq(XX)(Cq(A,X)). 
The functions <p and £ are called the almost automorphic component and the ergodic 
perturbation respectively of the function xp. Denote by ?Pi?^?(J,A r )(j?>W(J,A^) the set 
of all such functions xp. 

By Theorem 2.1.1 and Proposition 2. 1 .2 in [2], Jlfl(R, X) satisfies ( 1 .5H 1 .8). There- 
fore, JZ%(R,X) is a U r (R,X) space and ¥JVl(J,X) and JMA(J,X) are M\ r (J,X) spaces. 
It follows from Proposition 2.4 that 

2>.#fl(J,*) = JW(R, X)\j 0 <PW£^,X), 

= ^?(R,^)|j e G(J,*) 

and 

(5.i) #)c#j), IHI<M- 

THEOREM 5.2. The following statements hold. 

(1) A function £ € CRisin (PJ%P 0 (R)(Co(R)) if and only if is. 

(2) E G C(Ry is in ¥SVP 0 (R) n (Co(R) n ) if and only if the norm function |E(-)| is in 
fP^P 0 R(a(R) n ). 

PROOF. We show the theorem only for the case of TJVP 0 (R). The case of Co(R) is 
similar. 

(1) The sufficiency follows since 

I f t \m\/dx < i[ f t \m 2 *i l/2 i /_', i *] l/2 = t| £ icwi 2 &i l/2 . 

The necessity follows from the fact that (PS%Po(R) is an ideal of C(R)- 

(2) By(l),H = — .60 G iZflW if and only if G 2>-J*P 0 (R), * = 
1,2, • • • ,n. The latter is equivalent to that |E( )| 2 = £- =1 |€,(-)| 2 G &WP 0 (R), which, 
again by (1), is equivalent to that |H( )| G (PJ%P 0 (R). 

Let Q C C be compact and define 

<PWP Q {Q. x R) = TJVP 0 (R, C(n)), G(n x R) = &(R, C(0)) 

and 

JW(QxR) = j^(R,an)). 

A function ^ G Jlfl(n x R) is called almost automorphic in f G R uniformly in Z G O. 
PJWiQ x R)(j?J2Lq(Q x R)) is defined to consist of functions xp such that 

xp = if + t i i^p £ <M(n xRue ¥JVP 0 (n x R)(G(n x R))). 

The following lemma is straight forward. 
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LEMMA 5.3. Let <D *= {<p\ t <pi,- • ?«) G C(Rf- 7%e« <D € JW(R, C) //am/ only 
if® G AA(R) n . 

For H - (A|, A 2 » • • • . G , suppose that //(/) G n for all t G R. Define 

//xi:R- + flxRby//x = (/»,(/), MO, ' • • , M& t) (t G R). 

For*? = € JPJ^(R)"(jl^(R) n ), let d> = (</>,, <^ 2 , • • • ,¥>„) and 3 = 

* • • . 6.)> where and are the almost automorphic component and the ergodic 
perturbation respectively of t/>„ / = 1, 2, • • , n. 

The following theorem generalizes [26, Theorem 1.5] 

Theorem 5.4. Let $ g <PJl%(JX(n))(ww(j,C(nj))- If V e 
<PJW(Jr(SVW{Jr) and T(J) C Q, then i//o(f Xt )G ¥JWL(J)(JUM(J)). 

PROOF. It is sufficient to prove the case J = R,t/> G 2>.3J?(J,C(Q)), and ¥ 6 

LetV = y + Cand^ = <D + E with O = (y?i,</> 2 ,- • • ,<P») € ^?(R) n , as above. Since 
¥(0 6 Q when / G R, it follows from (5.1) that <D(/) G fi for / G R. Note that 

V>oOFxt) = (^oC^xO+^oOPxt) = ^o(<DxO+[¥>o(¥xi)-^o(<DxO+£oCFxi)]. 

We show that </? o (O x t) G -flL3(R). Let £ > 0. Since <p G C(Q x R), there is a 
5,e/2 >5 > Osuch that 

(5.2) |*<Zi.0-¥>(2a»0l<| (Z,,^€a|Zi-2b|<«;l€RX 

Note Lemma 5.3 and the hypothesis that <D G JW(R) W and p G .&fl(R,C(Q)), we 
conclude (v?,0) G -*W(R, C(Q) x R"). From the identity 

(5 3) + t) ' ' + T ) ~ 0 = ^(° (/ + T) ' ' + T ) 

- <^(0(r), / + t) + <^(<D(/), r + r) - v(O(0. 0 

we conclude N, (</?, O)) C (pd»xt)). This implies <p o (<D x t) e JW(R) 

by Proposition 1.1. 

To finish the proof, we need to show that the function h = <p o (T x t) — </? o (O x t) + 
C o (H* x i) is in ?Pj?!Po(R). First we show that o (¥ x t) - tp o (O x t) € fP-flrPoW- It 
is trivial in the case that tp = 0. So we assume that <p ^ 0. Let e and £ be as in (5.2). Set 

(5. 4) C 6 = {teR: \W(t) - O(0| = |H(/)| > 6}. 

It follows from [27, Definition 3 and Proposition 4] that there is a T > 0 such that when 
t> T 

(55) m([-M]nQ) < £ 



2/ -4|M|' 
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where m stands for the Lebesgue measure on R. It follows from (5.2), (5.4) and (5.5) that 
j t j' \^{s),s) - ^s\s)\ds 

Therefore, p o (V x t) - <p o (<D x t) G ¥JVP 0 (R). 

A little modification of the proof for [26, Theorem 1.5] shows that £ o (T x i) G 
?P^P 0 (R)- The proof is complete. 

Theorem 5.5. Consider systems of (he form 

dY 

(5.6) — = AY+V, 

dx 

where A = (a, y ) is a complex n x n matrix and ¥ = (i/>i , t/> 2 , • • • , V*)' € ?P^?(R)" 
(jlflJ^Ry). 7/" matrix A = (a,y) Aas wo eigenvalues with real part zero, then system 

(5.6) admits a unique solution Y = (y u y 2i • ■ • ,y n )' G f PAA(R) n (JVM(R) n ). Moreover 

(5.7) in<*m 

where K> 0 depends only on the matrix A. 

PROOF. By a discussion in [ 1 1 , Theorem 4.2] the matrix A can be considered trian- 
gular. Therefore the theorem follows by applying Theorem 4.5 n times. 
Now, consider a system of the form 

dY 

(5.8) — =AY + V + pGo(Yxl\ 

dx 

where p € C \ {0}, A is a complex n x n matrix, ¥ G iPJWCRy^JWJW 1 ), and 
G € 2>.*L3(Q x R)" (J1^(Q x R)"). Such a system is called quasi-linear. We get the 
generating system of (5.8) by putting /j. = 0. 

As in the proof for [26, Theorem 2.3], by using Theorems 5.4 and 5.5 one shows the 
following theorem. 

THEOREM 5.6. Let H> and A be as in Theorem 5.5. Let Y^ be the unique solution in 
£^(R)"(jWLfl(RY') of the generating system of (5.8), let a > 0 and let Q = \J{Z G 
C:\Z- J<°)(0| < a, t € R}. Assume that 

(1) G G <PJM(Cl x R)" (SWKfl x R)") such that 

(5. 9) ICC*, /) - G(Z", r)| < |zj - <|, (Z', Z" 6 a/ € R), 
vv/iere L > 0; 

(2) 0 < < min{l/IK,a/K||G||}, w/iere AT > 0 m (5.7; depends only on the 
matrix A. 

Then there exists a unique solution Y = (y\,yi, • • • ,y„)' € 21PL3(R)" (A%A(R) n ) of the 
system (5.8) such that Y(x) G Clfor all x G R. Furthermore, \\ Y - Y^\\ — 0 as n -» 0. 
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6. The Solutions of non-linear Parabolic Partial Differential Equations. For 

T > 0 and A > 0, let A = R x [0, T] and A, = A x [-A,A]. Consider the non- 
linear parabolic partial differential equation 

(6-1) g = ^ + V;(jM , M) ((,,/)€ A), 

where V> G C(A|) satisfies the Lipschitz condition 

(6. 2) |#t, /, v/) - rjj{x, t, v")| < l\v> - v"| (fr /) e A, v', v" 6 HMD- 

In this section, we shall use some knowledge from ordinary differential equations to 
establish some properties of solutions of (6.1). For this purpose, we first consider the 
following ordinary differential equation 

(6.3) g_ a 2 >; = r> 

where a > 0 and r € C(R)- Note that (6.3) admits a unique bounded solution given by 

(6.4) y 0 (x) = f e- al r{t)dt + e- m f x e al rit)dt^ 
for which 

(6-5) W<^IM|. 

It follows from Theorem 4.3 that^o in (6.4) is in (PJWl(R)( JWl(R)) if r is. 
Let us now consider the system of ordinary differential equations 

(6-6) ^ = rV-"*-.J + t/>(*,'*,"*-.), ft- 1.2.—.* 

where h = T / n € R, = kh and uq(x) = u(x, 0), where u is a solution of (6. 1 ). We need 
to consider (6.6) for variable n; note that the partition {/*}* =0 of [0, T] depends on n, as 
do all the functions in a solution (tfi,*2* * " . "«) of (6.6). 

As in the proof for [26, Lemma 3.1], one shows the following lemma. 

Lemma 6.1. Let ip e (PJZ%(A\)(JWl(Ai) satisfy the Lipschitz condition (6.2). 
Suppose u is a solution of equation (6. I) such that \\u\\ < A, u and du/dt are uniformly 
continuous, and u 0 = m(-,0) € 2lflLfl(R) (JVW(R)). Then there exist no e N and 
fV>0 such that (6.6) has a unique solution {u x ,u 2 , •••,«„) 6 <PJW(R) n (Jl?L?l(R) n ) 
for n > no; it satisfies 

0) \\u k \\ <A,\<k<n,and 

(2) the functions e 0 = 0, = u(-, t k ) — m* are in C(R) with 

(6-7) IMI<»M*X 
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where h = T/n, t k = kh, and u) is the modulus of uniform continuity of u and du/dt. 

THEOREM 6.2. Let t/> and u satisfy the conditions of Lemma 6.1. Then u is in 
fP^A)(jW#(A)). 

Proof. We only show the assertion that u G !P^W(A). Similarly, one shows the 
assertion that u G JlflL#(A). Let no be as in Lemma 6. 1 , and fixed n > no and / G [0, 7*J. 
Then there is a ko < n such that |/ — /*J < h. Recall that h = T/n and t k = kh. If 
B„ = {u k : k = 1, 2, •••,«} C (PJZ%(R) is the solution of (6.6) given by Lemma 6. 1 , the 
uniform continuity of u gives 

W{x, t) - «*o(*)| < W*. ') - 'Ol + W*. '*>) ~ C*)l 

<(1 + »XA) (*GR). 

It follows that the function u(-,t) is in the norm closure of (J^^ B n ; hence w(-,/) € 
PJW(R) (Proposition 2.4). 

Let ¥?(-, 0 and £(■, 0, V* and be the almost automorphic components and the ergodic 
perturbations respectively of «(•, /),«*,* = 1,2, • • To show that m € ¥JZ%(A), we 
need to prove that <p G ^W(A) and £ G fP^ 0 ( A). 

Since «(•, t!), «(•, f") G fP^R) for any />, r" € [0, T], so is u(-, 0 - «(-, f") and also 
«(•, /) - m* for / G [0, r] and k = 1, 2, • • , n. It follows from (5. 1 ) that 

(6.9) lk(-,' / )-^(-y , )ll<IK,/ , )-"(-y')ll 

and 

(6.10) \\<p(;t)-<Pk\\<\\u(;t)-u k \\ k= 1,2,.-,*. 

We show that <p G ^?(A). Let c > 0. Choose /i > no such that /i = T/n implies 

(6.11) 4(ff+lM/>)<£- 

Let <D = {<p { , if 2 , ■ ■ • , </>„}• B y Lemma 5.3, <D G A3(R, C). Let N > 0. It follows 
from (6.7), (6.9), (6.10) and (6.1 1) that, for |jc| < N,t G £(e/2, #,<!>) and / G [0, 7], 
there exists a such that |r - t^ | < A and 

\<p(x + r, 0 - <^(x, 0| < |V>CX + r, t) - <p{x + r, <J| + |<^(x + r, /*„) - <^(x + r)| 

+ K(* + t) - <Pko(x)\ + |^(x) - <^r,^)| 
+ M*,'*o)-^.')l 

<H«(-,0-iK-,MI + Ns^)-«i»ll 

+ lv*o(^ + r) - <fko(x)\ + ||m(-,/*o) - kJ| 

+ H-, /*«)-«(•, /)|| 
< 2(W+ IMh) + \<Pko(x + T) - ^(x)| 
<i + |<I>(x + r)-<DW|< £ . 
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It follows that E{t/2,N,<t>) C E{e,N,ip). This implies that tp is almost automorphic 
function by Proposition 1.1. 

As in the proof for Theorem 3.3 in [26], one shows that £ G (PJ^%{A). The proof is 
complete. 

Acknowledgment. The authors would like to thank Professor P. Milnes for his 
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THE 3jc + 1 CONJUGACY MAP 
DANIEL J. BERNSTEIN AND JEFFREY C. LAGARIAS 



Abstract The 3x + 1 map T and the shift map S are defined by T(x) = (3jc + 1 )/ 2 
for x odd, T(x) = x/2 for x even, while S(x) = (x - l)/2 for x odd, S(jt) = x/2 for 
x even. The 3jt + 1 conjugacy map 0> on the 2-adic integers Z2 conjugates S to 7", i.e. 
OoSocJ)" 1 b 7". The map <J> mod 2" induces a permutation Q>„ on Z/2"Z. We study 
the cycle structure of <t>„. In particular we show that it has order 2"~ 4 for n > 6. We 
also count 1 -cycles of for n up to 1000; the results suggest that <D has exactly two 
odd fixed points. The results generalize to the ax + b map, where ab is odd. 

1 . Introduction. The 3x + 1 problem concerns iteration of the 3x + 1 function 

j(3x+l)/2 if*=sl(mod2) 
*W \x/2 ifx = 0(mod2) 

on the integers Z. The well-known 3jc+ 1 Conjecture asserts that, for each positive integer 
n, some iterate 7*(n) equals 1, i.e., all orbits on the positive integers eventually reach the 
cycle {1,2}. 

The 3x + 1 function (1.1) is defined on the larger domain Z2 of 2-adic integers. It is 
a measure-preserving map on Z2 with respect to the 2-adic measure, and it is strongly 
mixing, so it is ergodic; see [8]. More is true. Let S: Za — ♦ Z2 be the 2-adic shift map 
defined by 

i.e., 5(Ego^ 2 ') = ££0 ^+> 2< ' if each b i >s 0 or 1. Then T is topological^ conjugate to 
S: there is a homeomorphism O: Z2 — » Z2 with 

(1.3) OoSoO- 1 = T. 

In fact T is metrically conjugate to 5: one map O satisfying (1.3) preserves the 2-adic 
measure. Thus T is Bernoulli. 

The map O is determined by (1.3) up to multiplication on the right by an automor- 
phism of the shift S. It is known that the automorphism group of S is isomorphic to Z/ 2Z, 
with nontrivial element V(x) = — 1 — x. (See [6, Theorem 6.9] and the introduction to 
[3].) We obtain a unique function O by adding to (1.3) the side condition <D(0) = 0. 
We call <D the 3x + 1 conjugacy map. This function has been constructed several times, 
apparently first in [8], where <D~' is denoted Qoo, and also in [1], [2]. 
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An important property of d> is that it is solenoidal. Here we say that a function / on 
Z2 is solenoidal if, for every n, it induces a function mod 2", i.e., 

x=y (mod 2") =/(y) (mod 2"). 

This solenoidal property, together with 0(0) = 0, implies that 

(1.4) <t>(x) = x (mod 2). 

For completeness, we give a self-contained proof that O is unique. Let <J> and O' be two 
invertible functions satisfying (1.3) and (1.4). Write Q and Q for their inverses. Then 
So Q = Qo Tand So Q' = Q o T, and (1.4) gives Q = Q' (mod 2). If0 = Q 1 (mod 2*) 
then Qo T = Q o T (mod 2*), so So Q = So Q 1 (mod 2 k ). Now So g and So Q' agree in 
the bottom k bits, and £) and Q 1 agree in the bottom bit, so Q and Q agree in the bottom 
k + 1 bits. Hence Q = Q (mod 2* +l ). By induction Q = Q? (mod 2 k ) for every A:, so 
Q=Q', sod> = 0'. 

There is an explicit formula for (D" 1 ([8]). Let T denote the m-th iterate of T. Then 

(1.5) Q-\x) = f)(r"(x) mod 2)2'. 

«=o 

This implies (1.3) and (1.4), and also shows that <D~' is solenoidal. 
There is also an explicit formula for <D ([2]). For* G Z2, expand x as 

/ 

in which {<//} is a finite or infinite sequence with 0 < d\ <d 2 < Then 

(1.6) <D(x) = -E3- / 2 </ '. 

/ 

This also implies (1.3) and (1.4), and shows that O is solenoidal. 

Various properties of the 3x + 1 map under iteration can be formulated in terms of 
properties of O. The 3x + 1 Conjecture is reformulated as follows ([2], [8]). Here Z + 
denotes the positive integers. 

3x + 1 Conjecture. Z + c <D(|Z). 

Furthermore, it is known that <D(Q Pi Z2) C Q n Z2. (This is easily proven from ( 1 .6); 
see [2].) The following conjecture is proposed in [8]. 

Periodicity Conjecture. <D(Q n Z 2 ) = Q n Z2. 

This would imply that the 3jc + 1 function T has no divergent trajectories on Z. Recall 
that a trajectory { T*(n) : k > 1 } is divergent if it contains an infinite number of distinct 
elements, so that \l*(n)\ -» 00 as k -» 00. In fact, if 



_|(3x + *)/2 ifx=l(mod2), 
x/2 if x - 0 (mod 2), 
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then the Periodicity Conjecture is equivalent to the assertion that, for all k = ±1 (mod 6), 
the 3jc + k function has no divergent trajectories on Z. (This follows from [9, Corol- 
lary 2.1b].) 

This paper studies the 3x + 1 conjugacy map O for its own sake. The function <1> is 
a solenoidal bijection; it induces permutations 0„ of Z/2"Z. Our object is to determine 
properties of the cycle structure of the permutations 0>„. In effect, our results give infor- 
mation about the iterates 0>* of O. We prove in particular that <D„ contains three "long" 
cycles of length 2"" 4 , for all n > 6. 

We remark that the results we prove are not related to the 3jc + 1 Conjecture in any 
immediate way; indeed for the iterates 7* the conjugacy (1 .3) gives O o 5* o OH =7*, 
a relation which does not involve O* for any k > 2. We do note that the Periodicity 
Conjecture is equivalent, for any k > 1, to the assertion that <I>*(Q n Z2) = Q n Z2. 
Consequently information about <P* may conceivably prove useful in resolving the Pe- 
riodicity Conjecture. 

The contents of the paper are as follows. In Section 2 we give a table of the cycle 
lengths of <t>„ for n < 20. This table motivated our results. We also give data on 1 - 
cycles of <J>„ for n < 1000. We conjecture that O has exactly two odd fixed points. In 
Section 3 we formulate results on the progressive stabilization of the "long" cycles of 
O n . In Section 4 we generalize these results to the conjugacy map for the ax + b function 



where ab is odd. We prove all these results in Section 5. The proofs are based on Theo- 
rem 5.1, which keeps track of the highest-order significant bit in the orbit of x mod 2" +2 . 
In Section 6 we reconsider "short" cycles of O n , and present a heuristic argument that 
relates their asymptotics to the number of global periodic points. This heuristic is con- 
sistent with the data on 1 -cycles presented in Section 2. 

There are two appendices on solenoidal maps. Appendix A shows the equivalence 
of "solenoidal bijection," "solenoidal homeomorphism," and "2-adic isometry." Appen- 
dix B shows that a wide class of functions U generalizing the 3x+ 1 map Tare conjugate 
to the 2-adic shift S by a solenoidal conjugacy map Oy. 

Finally, we note that, for odd k, the map Q(x) — kx conjugates the 3x+ 1 function to the 
3x+k function; i.e., Q0T0 Q~ x = 7^. Thus the cycle structure of the permutations mod 
2" of all the conjugacy maps $ 3> * are identical. Other properties of the 3x + 1 conjugacy 
map appear in [2], [10], [1 1]. In particular, O and O -1 are nowhere differentiable on Z2; 
see [10], [2]. 

We thank Mike Boyle and Doug Lind for supplying references concerning the auto- 
morphism group of the one-sided shift, and the referee for helpful comments. 

2. Empirical Data and Two Conjectures. By ( 1 .4), 0„ takes odd numbers to odd 
numbers. Let 0„: (Z/2"Z)* (Z/2"Z)* denote its restriction. The properties of $„ are 
completely determined by 0„. Indeed, 0(2^) = 2/<t>(x) by (1.6), so the action of <!>„_, 
describes the action of 0>„ on odd numbers times 2 j . 




(ax + b)/2 ifjc=l(mod2) 
x/2 if x = 0 (mod 2), 
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order(<D„) 




1 

2 
2 
4 



{ 1 9, 5 1 } {27, 59} {33, 53 } {39, 55 } {4 1 , 6 1 , 57, 45} 



4 



Table 2. 1 . Cycle structure of <j>„, n < 6. 1 -cycles are omitted. 



Each <t>„ consists of cycles of various lengths, all of which are powers of 2. (See 
Section 3 for a proof.) The exact form of 4>„ for n < 6 appears in Table 2. 1 . 

Table 2.2 below lists the number of cycles of various lengths in 4> n for n < 20. Let 
X„j denote the set of cycles of Q>„ of period 2 j , and let \X n j\ be the number of such cycles. 
From Table 2.2 we see, empirically, that 



We also see a progressive stabilization of the number of "long" cycles in <D„. In Sec- 
tions 3-5 we prove both these facts. 

How does 1^1, the number of cycles of <&„ of size 2 j , behave as n — > oo, for fixed /? 
We give data for the simplest case \X nS >\ of 1 -cycles. Table 2.3 gives all values of \X„ fi \ 
for n < 100, and Table 2.4 gives values of \X nfi \ at intervals of 10 for n < 1000. We 
computed the values lA^ol recursively for increasing n by tracking each 1 -cycle individ- 
ually. 

The tables show that |A" n> o| behaves irregularly, but has a general tendency to increase. 
In Section 6 we present a heuristic model which suggests that 

(2. 2) |A;,o| ~ F 0 n as n > oo, 

where Fq is the number of odd fixed points of O. Comparison with Table 2.4 suggests 
the following conjecture. 

FIXED POINT CONJECTURE. The 3x + 1 conjugacy map O has exactly two odd fixed 
points. 

We searched for odd rational fixed points, and immediately found two: x = - 1 and 
x = 1/3. The conjecture thus asserts that these are the only odd fixed points of O. We 
do not know of any approach to determine the existence or non-existence of non-rational 
odd fixed points. 

More generally we propose the following conjecture. 



(2.1) 



order(<D w ) = 1 



,n-4 



n >6. 
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0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 



2 
2 
4 
6 
6 
8 
14 
14 
10 
12 
16 
26 
22 
18 
20 
18 



1 

2 
3 
7 
10 
17 
21 
35 
40 
48 
53 



1 

3 
3 
8 
18 
24 
37 
70 
79 
63 111 



3 
0 



4 
14 
18 
23 
60 
98 

81 129 153 



3 
0 
2 



12 
16 
24 
50 
84 

96 179 186 137 



3 
0 
2 
10 
11 



3 
0 
2 
10 
11 
11 



31 
61 



3 
0 
2 
10 
11 
11 
29 



14 
40 

78 

91 242 236 207 131 
12 104 305 308 312 192 105 
16 86 375 401432 307 152 
26 95 424 573 564 445 281 133 
Table 2.2. Number of cycles \X Hj 



56 
99 



3 
0 
2 
10 
11 
11 
29 
54 
911 



3 
0 
2 
10 
11 
11 
29 
54 



3 
0 
2 
10 
11 
11 
29 



3 
0 
2 
10 
11 
11 



3 
0 
2 
10 
11 



3 
0 
2 
10 



3 
0 
2 



3 
0 



)f<fr n of order 2>,0<y<n. 



3jc + 1 CONJUGACY FlNITENESS Conjecture. For each j > 0, the 3jc + 1 conjugacy 
map O has finitely many odd periodic points of period 2f. 

We have no idea whether the 3x + 1 conjugacy map O has finitely many odd periodic 
points in total. There are examples of ax + b conjugacy maps that have no odd periodic 
points; see Section 4. 

3. Cycle structure of <D„: Inert Cycles and Stable Cycles. There is a simple re- 
lation between the cycles of <D„ and those of <D n+ i : For jc G Za, the cycle o^ x {x) that x 
belongs to in <D„ + i has length Ict^i (x)\ either equal to or double the length of the cycle 
o„(x) that x belongs to in <D„. 

This follows from a more general fact. Call a function .Z/m^'Z Z/m^'Z 
consistent mod m n if it induces a function /„ from Z/m"Z to Z/m"Z, i.e., if 

(3. 1) x, =x 2 (mod m n ) =»/^i(jci) =/^i(j: 2 ) (mod m"). 

Lemma 3. 1 . Letf^ x : Z/m n+1 Z Z/nf^Z be a function which is consistent mod 
m". If x is a purely periodic point off n+l then x is a purely periodic point of f„ and 

|0*,C*)| = k\cr„(x)\ 

for some integer k with 1 < k < m. 
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26 36 


56 


96 


124 


110 
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4 


4 


22 38 


54 


106 


124 


112 


108 


128 


92 






5 


6 


18 36 


54 


116 


114 


106 


114 


128 


96 






A 


A 
U 


20 36 


54 


on 


198 


09 


1 39 


1 36 


06 






7 

/ 


o 
o 


18 50 


68 


89 

8Z 


1 1 8 
1 1 8 


1 HA 

1UO 


140 


124 


i n9 
1 wz 






o 
O 


1 A 


12 60 


68 


Q9 

7Z 


OA 
V4 


1 1 A 
1 10 


144 


118 


ins 

1U8 






Q 


1 A 
14 


16 62 


84 


m9 

1UZ 


Q9 
VZ 


1 99 
1ZZ 


144 


104 


88 
88 






in 

1U 


in 

IK) 


26 50 


92 


ins 

1UO 


inn 

1UU 


1 79 


144 


98 
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Table 2.3. N 


umber of 


1 -cycles 


- 

in <D 10/+ *. 










(It A 


n 

u 


1 

1 


2 


3 


A 


j 


A 




7 


a 

0 


Q 


1 
1 


1U 


OA 


380 


700 


844 


1 978 
1Z /B 


1 078 
1U /8 


1330 


1 044 
1 V44 


9070 


Z 


zo 


on 


458 


788 


841) 


I 1 7 A 

I I /O 


1 1 7A 
1 1 JU 


1142 


Z 18U 


7 1 A7 

Z10Z 


1 

J 




1 1 A 
1 10 


452 


916 


1 1 74 
1 1 J4 


mnn 


1919 
1Z1Z 


1170 


9104 
Z I V4 


9970 
ZZJU 


4 


92 


156 


544 


780 


942 


914 


1270 


1240 


2226 


2128 


5 


108 


240 


574 


678 


874 


998 


1462 


1346 


2130 


2206 


6 


100 


278 


588 


908 


910 


1110 


1476 


1538 


2294 


2362 


7 


132 


282 


628 


818 


866 


1172 


1360 


1562 


2204 


2354 


8 


144 


320 


634 


784 


932 


1172 


1358 


1778 


2184 


2362 


9 


98 


378 


784 


870 


1060 


1072 


1190 


1974 


2114 


2242 


10 


90 


404 


714 


892 


1150 


1086 


1208 


1808 


2056 


2308 



Table 2.4. Number of 1 -cycles in ^looy+io*- 



Proof. The image of <Xn+i(x) under projection mod m" consists of k copies of a 
purely periodic orbit a„(x), for some k > 1. The bound k < m follows because any 
element of Z/m"Z has only m distinct preimages in Z/m"* 1 Z. ■ 

Lemma 3. 1 applies to <D„+i , because <D is solenoidal. Since m = 2 we have 

\a^{x)\ = k\a n {x)\ with*=lor2. 

We call a cycle <t„+\(x) split if |ct„+i(x)| = |<7 n (x)|, because a„{x) lifts to two cycles 
mod 2" +1 , namely o„+\(x) and a„^.\{x) + 2". If |<t,h-i(x)| = 2 \a„(x)\ we call <Tn+i(*) »«er/, 
because o„(x) has lifted to a single cycle. If ct„+i(*) is an inert cycle, and \a„(x)\ = p, 
then |ct„+i(jc)| = 2p and 

(3. 2) 0^,(x) = x + 2" (mod 2 B+I ). 

By induction on n, the length of any cycle \a„(x)\ is a power of 2. 

We call a cycle ct„(x) stafc/e if a m (x) is an inert cycle for all m > n. If is a stable 
cycle, then 

|* M (x)|= 2*-^' |a n _ ,(*)!, m>*. 
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For a stable cycle o n (x), Lemma 3. 1 guarantees that the map <D restricted to 



has no periodic points. 

Our main result concerning O is as follows. 

THEOREM 3.1. For the 3x + 1 conjugacy map <D, suppose that \o„(x)\ > 4 and that 
o„(x) and o n +\{x) are both inert cycles. Then o„ +2 (x) is also an inert cycle. Consequently 
a n (x) is a stable cycle. 

Theorem 3.1 follows from Corollary 5.1 at the end of Section 5. 

The hypothesis |cr„(*)| > 4 is necessary in Theorem 3.1. For example, <xs(3) = {3}, 
so both <t 6 (3) = {3, 35} and o 7 Q) = {3, 99, 67, 35} are inert, but ct 8 (3) = {3, 227, 195, 
163} is split. 

Corollary 3.1a. order(6„ ) = order(0„ ) = 2 n '\ for n>6. 

Proof. ct 6 (5) = {5, 1 7, 37, 49} is stable. ■ 
We next consider Table 2.2 in light of Theorem 3.1. Again let X nj - denote the set of 
cycles of <t>„ of period 2 / . Call X„j stabilized if it consists entirely of stable cycles. 

COROLLARY 3.1b. Assume that all X„ ( „_, are stabilized for 0 < j < k - \, and 
that\X HJI - k \ = lA^,^-,-*! = \X^2^2~k\- Then X m/n . k is stabilized for m > n, and 

\x m , m - k \ = Mf„.»-*l- 

This criterion gives the stabilized region indicated in Table 2.2. For n = 20 over 90% 
of all elements in (Z/2"Z)* are in stable cycles. 

4. The ax + b Conjugacy Map. Consider now the ax + b function 



where ab is odd. See [4], [5], [7], and [12] for various properties of T ttJt under iteration 
on Z. 

The 2-adic shift map S is conjugate to the general ax + b function T a>b by the ax + b 
conjugacy map O^: Z 2 — ► Z2; i.e., <t> aJ) 0S0 <D~i = T aJb . If jc = Yj 2 d ', where {d,} is a 
finite or infinite sequence with 0 < d\ < d 2 < • • •, then 



see [2]. Associated to <& ajb are the permutations <D aA „ on Z/2"Z obtained by reducing 
<D fl ,i mod 2". The following result generalizes Theorem 3.1. 



{yeZ 2 :y = Xj (mod 2") for some jc, G o„(x)} 



(4.1) 




(4.2) 



<X> a , b {x) = -bY J a- l 2 d >; 
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THEOREM 4. 1 . For the ax + b conjugacy map <b a ,b, suppose that a cycle a„(x) of 
®a,bst has \o n (x)\ > 4. 

(i) If a = 1 (mod 4), and a„(x) is an inert cycle, then a„+\ (x) is an inert cycle. 

(ii) If a = 3 (mod 4), and o„(x) and o„+i(x) are both inert cycles, then cr„+2(*) is an 
inert cycle. 

This theorem follows from Corollary 5.1 in Section 5. The proof actually shows that 
in case (i) the weaker hypothesis \o n (x)\ > 2 suffices, when 6 = 3 (mod 4). 

There are examples of ax + b conjugacy maps for which all cycles eventually 
become stable. Such <b at b then have no odd periodic points. Using Theorem 4. 1 we easily 
check that the 25x — 3 conjugacy map when taken mod 32 has an odd part consisting of 
two stable cycles of period 8. 

5. The Highest Order Bit Throughout this section, O = Q> aJ} is a general ax + 
b conjugacy map, where a and b are odd. We analyze the high bit of the iterates of 
O mod 2 rt+2 . All earlier results follow from Theorem 5.1 below. 

For x € Z2, expand x as 

00 

(5.1) x = £ bit*(x)2*, 

A=0 

where bit*(x) is either 0 or 1 . Define the bit sums 

(5.2) pop,(*):=£bit,0t). 

The ax + b conjugacy map is then given by 

00 _i. 

(5.3) o fl>fc W = E^)bit t (x)2*, 
by (4.2). 

Lemma 5.1. Ify, zeZ^ with z=y (mod 2"), then 

(5 4) 4>(z) - m - (z -y) = 2- (^i + pop„_ l( y)) 

•(bit n (y) + bit„(z)) (mod2 n+2 ). 

Proof. Expand <D(z) - <D(y) (mod 2 rt+2 ) using (5.3). We have bit A (z) = bit*0) and 
pop t (z) = pop k (y) for 0 < k < n — 1 , so the first n terms in 0(z) — <I>(y) cancel. Thus 

«u - m . 2»((^) bh. (Z ) - (^) biuw) 

+2 "*'((^) bit -< 2 )- (5^55) **««)• 
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Substitute a~ l = a (mod 4) in the coefficient of 2", and b = a~ l = 1 (mod 2) in the 
coefficient of 2"+ 

<D(z) - <D(y) = 2"(6aP°P.O') bit„(y) - ba™>' (z) bit„(z)) 
(5 ' 5) + 2^ (bit^,(z) - biWiM) (mod 2" +2 ). 

On the other hand 

(5. 6) z-y = 2"(bit„(z) - bit„(y)) + 2 n+, (bit ff+1 (z) - WWlO')) (mod 2" +2 ). 
Subtract (5.6) from (5.5): 

0(z) - <D(y) -(z-y)s 2"((6a p °P-W + l)bit„(y) - (W^W + l)bit„(z)) (mod V* 2 ). 

Substitute a* = l+(q—\)k (mod 4), pop A (x) bit*(jc) = (l +pop t _,(x)) bit*(;c), and then 
pop„-iC0 = pop„_,(y): 

<D(z)-0(y)-(z-y) 

= 2"((ft(l + (« - l)pop„(y)) + l) bit„(y) 

-(^i+C*-i)pon(o)+i)uu^) 

= 2"((nft + Dpop^ty)) bit„(y) 

- (a*> + 1 +fc(a - l)pop„_,(z)) bit„(z)) 

= 2"(ab + \+b(a- DpopW-i^XWMy) " bit„(z)) 

- 2*l(«^I + pop n _ l( y)) (bit n (v) - bit^)) (mod 2" +2 ). 

This is equivalent to (5.4). ■ 
Now fix x G Z2, and fix n > 0. Set |<t„(jc)| = 2/ and assume from now on that 

(5.7) <W*) is inert, 

so that |a„+i (x)\ = 2 /+1 . We wish to determine whether or not a^ix) is inert. According 
to (3.2) this occurs if and only if 

(5. 8) <t?*\x) = x + 2^ (mod 2 n+2 ). 
We now introduce the quantities 

e k [{\ := biu(0'(x)). 

In terms of the e k [f], we have 

(5. 9) a^x) is inert <=» [0] ^ [2> +l ], 

by (5.8). We proceed to evaluate e^[T x ] - e^O] mod 2. The main theorems of this 
paper are deduced from the following formula. 
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THEOREM 5. 1 . If\a„{x)\ = H and a„+i(x) is an inert cycle, then 

(5. 10) W^ 1 ] - WO] = 1 + ^2 + ^p±N (mod 2), 

where 

(5.H) N= EP0P n -,(^W). 

/=o 

PROOF. First we define X t = (0 ,+I+3, (x) - <D' +1 (*)) - («&**(x) - <D'(x)). Since 
an+Kx) is an inert cycle, O^Oc) = <D'(x) + 2" (mod 2 n+1 ), so, by Lemma 5.1, 

x, = 2*' (2^1 + ^ p°p-. (*«)) <™*> 2n+2 >- 



Adding up the A", gives 

2/-1 

E 

1=0 



(5.12) ^V iS2 - 1 (^iy + ^fi>w)(mod2^). 

.=n V 2 2 / 



Next define y, = 2"((e„[i + 1 + 2>] - e„[i + 1]) - (e„[i + 2>] - «?„[/])). The sum of the K, 
telescopes: 

E = 2"(e„[2' +1 ] - e^] - e.^] + e„[0]). 

i=0 

Since a^-i(x) is an inert cycle, e„[0] = e„[2 /+l ] ^ ^[2^], so 

(5. 13) E *i = 2"(2e.[0] - 2e„[2']) = 2* 1 (mod 2" +2 ). 

i=0 

On the other hand, 

X { - y, = 2" +1 fo* i [/+ 1 + 2>] - [i+ 1] - [i + 2>] + e*, [/]) 
ee 2 n+1 (e^, [/+ 1 + 2>] +«b*iP+H- «-i P + 2^ - *h KD- 
In this form the sum of X t — Y t also telescopes: 

E {Xi - *» = 2"* 1 ( e „ +1 [2^' ] - e„ +I [0]) (mod 2" +2 ). 

i=0 

Comparing this sum with (5.12) and (5. 13), we get 

2" +1 (e„ +1 [2' +1 ] - e«m = 2» +1 (^2> + ^^tf) - 2^ (mod 2"+ 2 ), 
which implies (5.10). ■ 
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Corollary 5.1 . (i) Ifa=\ (mod 4), then 

n/Hi rm - I 1 < mod 2 ) if b = 3 ( mod 4 > or > ^ 1 

] - e " +,[0]= (0(mod2) otherwise. 

0g) If a = 3 (mod 4), a/id <r„(jt) « me/-/, we/i 

* rt+h * rm _ | 1 (mod 2) ify > 2, 
c^lP l-^i[°]= ( 0(mo d 2)ify= 1. 

Note that (i) proves Theorem 4. l(i), and (ii) proves Theorem 4. l(ii), using (5.9). The- 
orem 3. 1 then follows as a special case of Theorem 4. l(ii). 

Proof, (i) Here a = 1 (mod 4), so the term involving N in (5. 10) drops out. 
(ii) Here a = 3 (mod 4), and j > 1, so (5. 10) simplifies to 

e„ +1 [^ +1 ]-e n+ ,[0]= 1+ W (mod 2). 

The inertness of a„(x) gives 

bit„_,(cD' +2M (x))= l-bit^'Ot)), 

so 

pop n _ x (V + *-\x)) + pop n _,(<D'(x)) = 1 (mod 2). 

Thus 

^^^'(pop^.^-'^+pop^^O'W)) = 1 =2'-' (mod 2). 

i=0 i=0 

Now (ii) follows. ■ 



6. Cycle Structure of 6„ : Short Cycles. We consider the behavior of "short" cy- 
cles of the 3jc + 1 conjugacy map; i.e., the behavior of \X nJ \ as n — * oo for fixed / We 
describe a heuristic model which relates the asymptotics of lA^I to the number of global 
odd periodic points of O. 

We first note that the odd periodic points Per*(<D) of <t> determine the entire set Per(<J>) 
of periodic points of O. The relation 

(6.1) <D(2x) = 2d>(x) 

implies that x has period 2/ if and only if 2x has period 2 j . Thus 

(6. 2) Per(O) = {2 k x : k > 0 and x G Per*(4>)}. 

Let Fj be the number of orbits of <D containing an odd periodic point of minimal period 
H. The 3jc + 1 Conjugacy Finiteness Conjecture of Section 2 asserts that all Fj are finite. 



Copyrighted material 



THE 3jc + 1 CONJUGACY MAP 



1165 



We obtain a simple heuristic model for the 1 -cycles X„, \ of <b„ by classifying them into 
two types: those arising by reduction mod 2" from an odd fixed point of O, and all the 
rest. Call these "immortal" and "mortal" 1 -cycles, respectively. Our heuristic model is to 
assume that each "mortal" 1 -cycle has equal probability of splitting or remaining inert, 
independently of all other 1 -cycles. When a "mortal" 1 -cycle splits, both its progeny in 
A'n+u are "mortal." An "immortal" 1 -cycle in X„\ always splits, and gives rise to two 
1 -cycles in AVh.i, at least one of which is "immortal." We also assume that only Fo 
"immortal" 1 -cycles appear in total, i.e., for all large enough n each "immortal" 1 -cycle 
splits into one "immortal" 1 -cycle and one "mortal" 1 -cycle. 

This model is a branching process model with two types of individuals. The expected 
number of individuals Z w l at step n is 

(6.3) E[Z„, l ] = F 0 n + c 0 , 

where Co is a constant depending on the levels of the initial occurrences of the fo "immor- 
tal" 1 -cycles. The empirical data in Tables 6.3 and 6.4 seem consistent with this model, 
with F 0 = 2. We know that F 0 > 2 in any case. The two "immortal" 1 -cycles that we 
know of both appear at n = 1 , so that if F 0 = 2, then c 0 = 0 in (6.3). 

To obtain a heuristic model for \X nj \ when j > 1, we use a refined classification of 
cycles of 4>„. A step consists of passing from 0>„_i to <&„. For 0 < d < j < n let X nj4 
denote the set of cycles of 4>„ of size 2 J which have remained inert for exactly d steps. 
Let Y nJfd denote the subset of X nj4 that consists of cycles that split in going to <b n +\ . Then 
we have 

n 

l&M/ol = 2 £ | J^l 

and 

|^t + l,/+U+l| = \XnjA ~ \YnjA ■ 

We know the following facts about these quantities: 

( 1 ) If a cycle of length at least 8 has been inert for d > 2 steps, it remains inert. Thus 
\YnjA =0if/>3and</>2. 

(2) Any cycle of length 4 which has been inert for d = 2 steps must split; i.e., 

|A"„,2.2| = | ^,2,2 1- 

(3) Any odd periodic point x of <I> of period 2 j gives rise to a cycle of period 2 / of 4>„ 
for all sufficiently large n. This cycle always splits. Such cycles are in both X n j$ 
and Y nJ fi. 

The quantity we are interested in is 

n 

\ X *j\ = X) \XnjA • 

The facts above imply that \X nJ \ is entirely determined by knowledge of \X m j,o\, | Y mJt0 \, 
and|y m>/ ,i|,forallw <n. 

Our heuristic model is then to suppose the following: 
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( 1 ) Each cycle in X n j t \ has (independently) probability 1 /2 of falling in . 

(2) Each "mortal" cycle in X„j fi has (independently) probability I /2 of falling in 
Y„jfi, and if so its two progeny in A^i^o are "mortal." 

(3) Each "immortal" cycle in X„j q lies in Y n j Q, and one of its progeny in JGi+i/o is 
"immortal" and the other is "mortal," with finitely many exceptions. 

This is a multi-type branching process model. If Z n j denotes the total number of individ- 
uals in such a process, then one may calculate that, for large n, 

(6. 4) £[Z W ,,] = X -F Q n 2 + (f, + ^F 0 )n - F, + l -F 0 + e u 

in which <?i is a constant depending on the initial occurrence of "immortal" cycles. (We 
assume that c 0 = 0.) For j > 2, where stable cycles may occur, the formula for E[Z nJ ] 
becomes quite complicated. 

It might be interesting to further compare predictions of this model for j > 1 with 
actual data for <t>. We know of one odd periodic cycle of O of length 2, namely {1,-1/3}; 
/.e.,<D(l)= -1/3 and <D(- 1/3)= l.ThusF, > 1. 

7. Appendix A. Solenoidal Maps. Call a map F: Z2 -» Z2 solenoidal if, for all n, 
(A.1) x = y (mod 2") => F(x) = F(y) (mod 2"). 

An equivalent condition in terms of the 2-adic metric || 2 is that F is nonexpanding; i.e., 

(A.2) \F(x)-P(y)\2<\*-y\2> Mx,y€Za. 

If F\ and F 2 are solenoidal maps, then so is F\ o F 2 . 

Call a family of functions F„: Z/2"Z — Z/2"Z compatible if F„ agrees with F fl _, 
under projection 7r„:Z/2 /I Z — > Z/2"" , Z; Le., if tt„ o F n = F„_i o ir n . A compatible 
family {F„} has an inverse limit F: Z2 — » Z2 defined by 

(A.3) F(x) = F n (x) (mod 2"), foraU/i. 

The term "solenoidal" is justified by the following lemma. 

Lemma A.l. F is solenoidal if and only if F is the inverse limit of a compatible 
family {F n }. 

Proof. If F is solenoidal, then F mod 2" induces a function F„: Z/2"Z -» Z/2"Z, 
for each n\ and {F„ } is a compatible family. The reverse implication follows from (A.3). ■ 

Lemma A.2. Let U be the inverse limit of a compatible family {(/„}. Then the fol- 
lowing are equivalent. 

(i) U is a bijection. 

(ii) For each n, U„ is a permutation. 

(Hi) For each n, ifU(x) = U(y) (mod 2") then x=y (mod 2"). 
Proof, (i) => (ii). U is surjective, so t/„ is surjective. 
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(ii) (i). Write V„ = U~ l . Then {V n } is a compatible family. Let V be its inverse 
limit. By construction U o V is the inverse limit of identity functions, so t/ o V is the 
identity. Similarly V o (/ is the identity. Hence £/ is a bijection. 



(ii) => (iii). If U(x) = U(y) (mod 2") then £/„(* mod 2") = U n (y mod 2") so 
xmod2" =^ mod 2". 

(iii) =» (ii). Suppose that U„(a) = U„(b). Select* and y in la such that a = x mod 2", 
6 = y mod 2". Then £/„(* mod 2") = £/„(y mod 2"), so {/(*) = £/(y) (mod 2"), so 



COROLLARY A.3 . The following are equivalent. 

(i) U is a solenoidal bijection. 

(ii) U is a solenoidal homeomorphism. 

(iii) U is a 2-adic isometry. 

U is a 2-adic isometry if | U(x) - U(y)\ 2 = \x- y\ 2 . 

PROOF. (i)=>(iii). U is solenoidal so \U(x) - U(y)\ 2 < \x-y\ 2 . On the other hand, 
by Lemma A.l, U is an inverse limit; and (/ is a bijection, so \U(x) - U(y)\ 2 > \x -y\ 2 
by Lemma A.2 (i => iii). 

(iii) (ii). Since | U(x) — U(y)\ 2 < \x — v| 2 , U is solenoidal. By Lemma A. 1 , U is an 
inverse limit; by Lemma A.2 (iii i), C/is a bijection. Since \U(x) - U(y)\ 2 >\x— y\ 2 , 
U~ x is solenoidal. Finally, solenoidal implies continuous. 

(ii) => (i). Immediate. ■ 

8. Appendix B. Functions SolenoidaUy Conjugate to the Shift For any two sole- 
noidal bijections V 0 , V x define Uy 0tVl : Zz -» Z2 by 



For example, take Vq(x) = xand V\(x) = ax+(a+b)/2; then Uv*,v x is the ax+b function. 

In this appendix we show that a map is solenoidally conjugate to the 2-adic shift map 
S — i.e., conjugate to S by a solenoidal bijection — if and only if it is of the form Uy 0 y x . 

Lemma B.l. Let V be a solenoidal bijection. If z = w (mod 2 m_1 ) then V(z) = 
V(w)+z- w (mod 2 m ). 

PROOF. If z = w (mod 2 m ) then V{z) = V(w) (mod 2 m ). 

If z = w + 2 m ~ l (mod 2 m ) then still V{z) = V(w) (mod 2 m ~ l ). By Corollary A.3, V 
is an isometry, so if V(z) = V(w) (mod 2 m ) then z = w (mod 2 m ), contradiction. Thus 
V(z) = V(w) + 2 m ~ x (mod 2 m ). ■ 

Lemma B.2. Set U = U Vo , Vi . Fix m > \.Ify = x + 2 m e (mod 2 m+l ) then U(y) = 
U(x) + 2 m - i e(mod2 m ). 

PROOF. Put b = x mod 2; then U(x) = V b (S(xj). Also U(y) = V b (S(y)) t since 
y = x (mod 2). We have 5(y) - 5(jc) + 2 m " , e (mod 2 m ); by Lemma B.l, V b (S(y)) = 



x = y (mod 2"), so a = b. 




V b (S(xj) +2 m ~ [ e (mod 2 m )- 



■ 
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Lemma B.3. Set U = Uy QyVx . Fix m > j > 1. Ify = x + 2 m e (mod 2 m+l ) then 
U{y) = U(x) + 2 m ~U (mod 2 m -> +1 ). 



LEMMA B.4. Set U = Uy 0 ,y r Fix m > 1. Ify = x + 2 m e (mod 2 m+1 ) then lT(y) = 
ir(x) + e (mod 2). 



LEMMA B.5. Set V = Uy 0 ,y t . Fix bo, by , b 2 , . . . G {0, 1 }. Define x 0 = 0 andx m+l = 
x m + 2 m (b m - *T(jr m )). Theny = x m (mod 2 m ) if and only if If (y) = b t (mod 2) for 
0 < i < m. 

PROOF. We induct on m. For m = 0 there is nothing to prove. 

Say y = x m+x (mod2- +1 ). Then y = x m + 2 w (fc m - lT(x m )) (mod2'" +, ); by 
Lemma B.4, lT(y) = lT(x m ) + b m - lT(x m ) = b m (mod 2). Also>; = x m (mod 2" 1 ), so 
by the inductive hypothesis lf(y) = bi (mod 2) for 0 < i < m. 

Conversely, say l/(y) = bi (mod 2) for 0 < * < m. By the inductive hypothesis 
y = x m (mod 2 m ). Write y = x m + 2 m e. Then fe m = tr(y) = ir(x m ) + e (mod 2) by 
Lemma B.4. Thus y = x m + 2 m (ft m - (T(jc m )) = x w+ , (mod 2 m+l ). m 

THEOREM B. 1 . Set U = Uy 0 , Vr Define Q(x) = T^ =0 (ir(x) mod 2)2 m . Then Q is 
a solenoidal bijection, and U = £T 1 o S o Q. 

Thus any map of the form Uy 0i y t is solenoidally conjugate to S. (See Theorem B.2 
below for the converse.) Q~ l generalizes the ax + b conjugacy map. 

PROOF. Injective: Say Q(y) = Q(x). Define b m = (T(x) mod 2; then U"(y) = 
ir(x) = b m (mod 2). Next define x 0 = 0 and x m + x = x m + 2 m (b m - t/"(jc m )). By 
Lemma B.5, y = x m (mod 2 m ) and x = x m (mod 2 m ). Thus y = x (mod 2 m ) for every /n; 

i.e.,y = x. 

Solenoidal: Say y = x (mod 2"). Define fc m = lT(x) mod 2, jco = 0, and jc m+ i = 
x m + 2 m (b m - Ifix^). Then x = x„ (mod 2") by Lemma B.5, soy = x n (mod 2"); by 
Lemma B.5 again, LT(y) = b m (mod 2) for 0 < m < n. Thus Q{y) = Q{x) (mod 2"). 

Surjective: Given b = E^ 0 W with b i e {°» 1}i define x 0 = 0 andx m+ , = x m + 
2 m (6 m - CT(jc m )). Since x m+! = x m (mod 2 m ) the sequence jci,x 2 ,... converges to a 
2-adic limit y, withy = x m (mod 2 W ). By Lemma B.5, lf"(y) = b m (mod 2) for all m. 
Thus 0(y) = ft. 

Finally, it is immediate from the definition of Q that QoU = SoQ. ■ 

Theorem B.2. Let Q be a solenoidal bijection. Define U = Q~ l o S o Q. Then 
U = Uy 0t y t for some solenoidal bisections Vq,V\. 

Proof. If 0(0) is even then Q~ \x)=x (mod 2) for all x; so write 



Proof. Lemma B.2 and induction on j. 



Proof. Lemma B.3 with j = m. 
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Then W 0 , W\ are solenoidal bijections, and U = Uv^v x where V, ■■ = Q o W t . 
Similarly, if 0(0) is odd then 0~ 1 (x) = - 1 - x (mod 2) for all x; so write 

, f 1 + 2W 0 (x/2) if x m 0 (mod 2), 
U W- j 2 fr,((x-l)/2) if Jc=l(mod2X 

Again PF 0 , are solenoidal bijections, and U = (/k 0 ,k, where = Qo W{. * 

References 

1. E. Akin, Why is the 3x + I problem so hard? Math. Magazine, to appear. 

2. D.J. Bernstein, A non-iterative 2-adic statement of the 3x+ 1 conjecture, Proc. Amer. Math. Soc. 121(1994), 
405^108. 

3. M. Boyle, J. Franks and B. Kitchens, Automorphisms of one-sided subshifts of finite type, Ergod. Th. Dyn. 
Sys. 10(1990), 421-M9. 

4. R. E. Crandall, On the '3* + 1 -problem, Math. Comp. 32(1978), 1281-1292. 

5. Z. Franco and C. Pomerance, On a conjecture of Crandall concerning the QX+ 1 problem, Math. Comp. 
49(1995), to appear. 

6. G. Hedlund, Endomorphisms and automorphisms of the shift dynamical system. Math. Systems Theory 
3(1969), 320-375. 

7. E. Heppner, Eine Bemerkungzum Hasse-Syracuse Algorithmus, Arch. Math. 31(1978), 317-320. 

8. J. C. Lagarias, 77ie 3x + 1 problem and its generalizations, Amer. Math. Monthly 92(1985), 3-23. 
9 , The set of rational cycles for the 3x + 1 problem, Acta Arithmetica 56( 1990), 33-53. 

10. H. MOller, Das '3n + 1' Problem, Mitteilungen der Math. Ges. Hamburg 12(1991), 231-251. 

11 , fiber eine Klasse 2-adischer Funktionen im Zussamenhang mit dem "3x+ 1 "-Problem, Abh. Math. 

Sem. Univ. Hamburg 64(1994), 293-302. 
12. R. P. Steiner, On the "QX+ 1 problem, " Q odd. I. II, Fibonacci Quart. \% 1981), 285-288, 293-296. 



AT&T Bell Laboratories 
Murray Hill, NJ 
USA 07974 

email: jcl@research. att. com 

current address: 
Department of Mathematics, 

Statistics, and Computer Science 
The University of Illinois at Chicago 
Chicago, IL 
USA 60607-7045 
email: djb@math.uic.edu 



Department of Mathematics 
University of California 
Berkeley. CA 
USA 94720 



Copyrighted material 



Can. J. Math. Vol. 48 (6), 1996 pp. 1170-1179. 



FINITE SUBGROUPS IN INTEGRAL GROUP RINGS 



MICHAEL A. DOKUCHAEV AND STANLEY O. JURIAANS 



Abstract. A p-subgroup version of the conjecture of Zassenhaus is proved for 
some finite solvable groups including solvable groups in which any Sylow /^-subgroup 
is either abelian or generalized quaternion, solvable Frobenius groups, nilpotent-by- 
nilpotent groups and solvable groups whose orders are not divisible by the fourth 
power of any prime. 



1. Introduction Let 11\ZG denote the group of units of augmentation one of the 
integral group ring of a finite group G. The Zassenhaus conjecture (ZC3) says that any 
finite subgroup of U\ZG is conjugate in QG to a subgroup of G (see [19, Chapter 5]). 
Its particular case (ZC 1 ) states that the same is true for torsion units of U\ ZG. We know 
that (ZC3) holds for nilpotent groups [22] and for split metacyclic groups ([15], [21]). 
K. W. Roggenkamp and L. Scott have shown that the Zassenhaus conjecture is false 
and a counterexample is a finite metabelian group [11]. However, somewhat weaker 
statements hold for large families of finite and infinite groups (see [ 1 9, Chapters 5 and 6] 
and [1], [3], [4], [7], [8], [9], [10], [12]). In the present paper we consider the following 
/'-subgroup version of (ZC3). 

, _ ZC3) If " is a / , - sub 8 rou P of ZG toe" mere exists a unit a G QG sucn 
V that a 1 Ha C G. 

In particular, if (p-ZC3) is true for a group G then any Sylow p-subgroup of UxZG 
is rationally conjugate to a /7-subgroup of G. Conjugation of those Sylow subgroups of 
'U] ZG which can be embedded into a group basis is investigated in [9], [10]. 

In this paper all groups G are assumed to be finite. In Section 2 we establish a reduction 
modulo a normal subgroup. We apply it to generalize a result of [16] and to prove (p- 
ZC3) for nilpotcnt-by -nilpotent groups. In particular, this conjecture is true for both 
metabelian and supersolvable groups. We also give a partial solution of Problem 32 of 
[19] and point out that (p-ZC3) implies a positive solution of that problem. In Sections 3 
and 4 we establish (ZC3) for S 4 and a covering group of it, the Binary Octahedral Group. 
We apply these results in Section 5 to prove (p-ZC3) for solvable groups in which any 
Sylow subgroup is either abelian or generalized quaternion. As a consequence we deduce 
(p-ZC3) for solvable Frobenius groups. We also prove (p-ZC3) for a family of groups 
including those solvable groups whose orders are not divisible by the fourth power of 
any prime. 
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2. A reduction step and some applications. For an element a = E geG oc(g)g of 
ZG we put a(g) = Ylhec a (h) where C g is the conjugacy class of g G G. We use symbol 
~ to denote conjugation in a group. 

Let N be a normal subgroup of G, G = G/N, ZG — » Z(G/N) the natural map, 
g = ^(g) forg G G. This notation will be used in all what follows. 

Lemma 2.1. Let a G WiZG fo? a tors/cm unit, 0 = ^(ar) a«</ (o(a), = 1. /^/Ae 
order ofgeG is relatively prime to \N\ then a(g) = 0(g). 

Proof. Set S g = {h G G : A ~ g in G} and = {A G S g : o(/i) = o(g)}. We see 
that 0(g) = T,hes g <*(h)- Note that if h is not in S g then (o(h), \N\) / 1 and consequently 
there is a prime p such that p | o(h) but p does not divide o(a). By [19, Lemma 38. 11], 
a(h) = 0. Since the complement of 51 in S g is a normal subset of G, we have that 
0(g) = E/,gsj oc(h). It suffices to show that the elements of S£ are conjugate to g. Indeed, 
if h e S' g then r x ht = gti for some teG,9e Af and the equality o(h) = o(g) implies that 
the cyclic subgroups (g) and (g6) are complements for N in (g). Since (o(g), \Nu = 1, 
we get, by Schur-Zassenhaus Theorem, that g9 is conjugate to g. The result follows. ■ 

The next result generalizes [7, Lemma 2.3]. 

Theorem 2.2. Let H be a finite subgroup of 1l\ ZG such that (|//|, \N\) = 1 and G 0 
be a subgroup ofG with (|G 0 |, \N\) = 1. Then H is rationally conjugate to G 0 iff*V(H) is 
conjugate to ^(Go) in QG. 

Proof. We only have to prove the converse. Let H = and G 0 = M'(Go). Let 
7 _1 //7 = G^ for some 7 G QG, a £ H and /3 be as above. We see that h a = 7~'^7 is, 
up to conjugacy, the unique element of G with 0(h a ) ? 0. From [19, Lemma 38.11] it 
follows that (o(/i a ), = 1. From the Schur-Zassenhaus Theorem it follows that we 
can choose g a G G such that h a = ^(ga) and (o(g a ), = 1. Then it follows from [19, 
Lemma 38.11] and Lemma 2. 1 that, up to conjugacy, g a is the unique element of G with 
a(g a ) f 0. Since 0Gb|, = 1 , the restriction of *F to Go gives an isomorphism between 
Go and Go. Denote by the inverse of this isomorphism and define a homomorphism 
4>: H— > G 0 by setting 4>(a) = 4 / i(7" , /?7). Since (o(<f>(aj), |/V|) = 1, Lemma 2.1 implies 

that a(<f>(a)) = 0(v<l>(a)) = 0(h a ) f 0 and <f>(a) is conjugate to g a . It follows by [19, 
Lemma 4 1 .4] that H is rationally conjugate to Go . ■ 

REMARK. We have proved that if H < U{ZG and (\H\, \N\) = 1 then 4> is injective 
on//. 

As a consequence we have the following: 

COROLLARY 2.3. Suppose that (ZC3) holds for the factor group Gj N. Then any finite 
subgroup He 11\ZG whose order is relatively prime to the order ofN is rationally 
conjugate to a subgroup of G. 

We also obtain some consequences for split extensions. 

COROLLARY 2.4. Let G be an extension of a nilpotent group N by a group X which 
satisfies (p-ZC3). If the orders ofN and X are relatively prime then G satisfies (p-ZC3). 
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Proof. Let H be a finite p-subgroup of U\ ZG. If p docs not divide the order of 
N then we use Theorem 2.2 and the assumption on X. If p does divide \N\ then G has 
a normal Sylow p-subgroup and hence, by [19, Theorem 41.12], we obtain that H is 
rationally conjugate to a subgroup of G. m 

We give now an improvement of Lemma 37.13 of [19]. 

LEMMA 2.5. Let G = N X X, where the orders of N and X are relatively prime, and 
let a = vw 6 U\ZG be a torsion unit with vE + A(G, M)) and w € U\ZX. If 
(o(a), |jV|) = 1 then a and w are rationally conjugate. 

Proof. We observe that a(g) = w(g) for all g e G. Indeed, if (o(g), \N\) f 1 then 
it follows from [19, Lemma 38.11] that ce(g) = w(g) = 0. If (o(g), \N\) = 1 then, by 
Schur-Zassenhaus Theorem, we may suppose that g GX and apply Lemma 2. 1 . 

Now let d be a divisor of o(a). Then of 1 = v d W* with v d € ll( 1 + A(G, A 7 )) and we 
use the same reasoning for the units a*, w d . Hence, according to [13, Theorem 2], a and 
w are conjugate in QG. ■ 

The next result is a modification of Lemma 37.6 of [19]. 

Lemma 2.6. Let H\ and H2 be isomorphic finite subgroups of U\ZG with a given 
isomorphism if. H\ — > // 2 . Suppose that \(h) = \{if{h)^ for all h G H\ and all absolutely 
irreducible characters \ of G. Then H\ is conjugate to H 2 in QG. 

Proof. We extend the representation T: G — » Gl(w, C) corresponding to x linearly 
to r\ .H\ — » Gl(/i, C). By assumption the characters of T| and Tup are equal and, 
consequently, the images of H\ and H2 are conjugate in any simple component of CG. 
Hence H\ is conjugate to H 2 in CG and Lemma 37.5 of [19] implies that the conjugation 
can be taken in QG. ■ 

Now we extend Theorem 37.17 of [19]. 

THEOREM 2.7. Let G be as in Lemma 2.5. Then any finite subgroup H of ft, ZG with 
(\H\, \N\)=lis rationally conjugate to a subgroup of tl\ ZX. 

PROOF. For a e H we write a = vw with v 6 fi( 1 + A(G, AO) and w E Z1\ZX. By 
Lemma 2.5 the isomorphism H 9 a = vw — ♦ w satisfies the hypothesis of Lemma 2.6. 
Hence H is conjugate to H 0 in QG, where H 0 is the image of H in lii ZX. m 

COROLLARY 2.8. Let N be a normal subgroup of G and H be a finite subgroup in 
1 + A(G, A/). If p is a prime which divides \H\ then p divides \N\. In particular, ifN is a 
Hall subgroup of G then \H\ divides \N\. 

Proof. We already know that \H\ is a divisor of \G\. Suppose that there exists a 
prime p that divides the order of H and does not divide |A^| . Let a G H be a unit of order 
p. Then ^(oe) = 1 and, by Theorem 2.2, we have that a is rationally conjugate to 1, a 
contradiction. ■ 

THEOREM 2.9. Let Gbea nilpotent-by-nilpotent group. Then (p-ZC3) holds for G. 
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PROOF. Let H be a p-subgroup of ^/i ZG and G\ be a normal nilpotent subgroup of G 
so that G/Gi is nilpotent. If Gi is not a p-group, then G possesses a normal p'-subgroup 
N. It follows from Theorem 2.2 and induction on the order of G that H is conjugate in 
QG to a subgroup of G. If Gi is a p-group, then the Sylow p-subgroup of G is normal 
and [ 1 9, Lemma 41.12] implies that H is rationally conjugate to a subgroup of G. ■ 

The proof of the following lemma can be found in [7] (see Lemma 1.5). 

LEMMA 2.10. Let G be a solvable group and P an abelian Sylow p-subgroup of G. If 
P is not normal in G then Gy(G) f 1. 

PROPOSITION 2.11. Let P be an abelian Sylow p-subgroup of a solvable group G. If 
H is a finite p-subgroup of 11\ ZG then H is rationally conjugate to a subgroup of G. 

Proof. By [ 1 9, Theorem 4 1 . 1 2] we may assume that P is not normal in G. It follows 
from the preceding lemma that N = 0^{G) ^ 1. Since the factor group G/N satisfies 
our hypothesis we can use Theorem 2.2 and induction to conclude that H is rationally 
conjugate to a subgroup of G. ■ 

S. K. Sehgal has proposed the following question [19, Problem 32]: 
Letue1l(\+ A(G, AO) be a torsion unit with N < G. Does o(u) divide \N\? 
Now we point out that (p-ZC3) implies a positive solution of this question. 

PROPOSITION 2.12. Let N be a normal subgroup of a group G which satisfies (p-ZC3). 
IfH is a finite subgroup of<U(\+ A(G, AO) then \H\ divides \N\. 

PROOF. Let H p be a Sylow p-subgroup of H. By (p-ZC3), H p is rationally conjugate 
to a subgroup H 0 of G. Going down modulo N we see that H 0 C N. Hence \H 0 \ divides 
\N\, and consequently \H\ divides \N\. ■ 

Note that Corollary 2.8 gives a partial solution of this problem. 

3. (ZC3) for 5 4 . The Zassenhaus conjecture for cyclic subgroups in ZS 4 was proved 
in [5]. In this section we prove the following: 

Theorem 3.1. (ZC3) holds for S 4 . 

Proof. Let G = S* and let H be a finite subgroup of Ux ZG. It is known that G 
has a faithful irreducible complex representation T: G — » Gl(3, C) such that the trace of 
r(( 1 2)) is 1 . We denote also by Y the extension of this representation to ZG. Since (ZC 1 ) 
holds for G it follows that T is faithful on H. Therefore 

(3.2) \nm\ = W- 

Denoting by F the Fitting subgroup of G we have that F = ((12)(34), (13)(24)) and 
G/F ^ 5 3 . Since F is abelian, there exists an invertible matrix X such that X X Y(F)X 
has a diagonal form. It is easy to see that 

(3 .3) X~ 1 IW = {/, diag(- 1,-1,1), diag(- 1,1,-1), diag( 1,-1,-1)}. 
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Denote by ¥ the natural map ZG -» ZG/F, H = ¥(//) and H 0 = f/p|(l + A(G, F)). 
In view of (ZC 1 ), going down modulo F, we obtain that 

(3.4) he H 0 if and only if 7 _1 /r7 G F for some unit 7 G QG. 

We may also assume that H is not cyclic. According to Lemma 2.6 it suffices to find a 
monomorphism ip:H-+G such that h ~ </?(/t) in QG for all h G H. We consider several 
cases. 

Case 1. H= (u,v) is isomorphic to the Klein four group. Since \H\ divides |G|, the 
order of H divides 6 and [H : Ho] = 1 or 2. 

If the index is 1 then, by (3.4), the map <p:H — > F defined by ip(u) = (12)(34), 
<p(v) = (13X24) is a group isomorphism such that h ~ ip(h) in QG for all h G H. Thus, 
H is rationally conjugate to F. 

Suppose now that [H : H 0 ] = 2. Choose generators u, v such that u £ H 0 and H 0 = (v). 
We have that u ~ (12) and v ~ (12)(34) in QG. Clearly uv £ H 0 and, therefore, 
uv ~ (12) ~ (34) in QG. We now define an isomorphism <p:H — ((12), (12X34)) by 
putting <^( M ) = ( 1 2), v>(v) = ( 1 2)(34). Then h is rationally conjugate to <p(h) for all h G H 
and consequently H is conjugate in QG to a subgroup of G. 

Case 2. The order of H is 8. Note that in this case [H : H 0 ] = 2. We show that H is 
not abelian. First suppose that H is elementary abelian and let u\, U2, «3 be generators 
of H such that H 0 = (m 2 ,m 3 ). There exists a matrix Y such that Y~ l r(H)Y consists of 
diagonal matrices. For h G H we put </(/i) = Y x T(h)Y. Note that K '/Zor consists of 
the diagonal matrices given in (3.3). So there is u e H Q so that d(u) = diag(l , -1,-1). 
Now since «, does not belong to H 0 we may suppose that d(u\ ) = diag(- 1,1,1). Hence 
d{uu\) = diag(-l, -1, -1), a contradiction since uu } is rationally conjugate to (12). 

Let H = <k,v), where o(u 2 ) = o(v) = 2. Note that u does not belong to H 0 and, 
consequently, // 0 is generated by u 2 and v. Let Y be such that Y' ] T(H)Y is in diagonal 
form. As above, the diagonal form of Ho consists of the matrices given in (3.3). Since 
m 2 G Ho we may assume that d(u 2 ) = diag(- 1,-1,1). Hence, d(u) = diag(±/, ±/', ±1). 
Choose w G H 0 so that d(w) = diag(l, — 1, — 1). The element uw has order 4 so, since 
(ZC1) holds for G, we see that uw is rationally conjugate to (1234). Hence, d(u) and 
d(uw) are conjugate. However, it is easy to check that the matrices diag(±/, ±1, db 1 ) and 
diag(±i, ±i, ± 1 ) diag( 1 , — 1 , — 1 ) are not conjugate in Gl(3, C), a contradiction. 

Thus H is not abelian and since Ho has exponent 2 we see that H must be isomorphic 
to the dihedral group of order 8. Let H = (m, v : u 4 = v 2 = 1, v -1 nv = u 3 ). Then u is 
not in Ho and we may choose v such that Ho is generated by u 2 and uv. By (3.4), the 
nontrivial elements of H 0 are conjugate to (12)(34). Since (ZC1) holds for G we have 
that the other elements of order 2 are rationally conjugate to (13) ~ (24) and those of 
order 4 are conjugate to ( 1 234). Put H t = ((1 234), (13)) and define an isomorphism ofH 
to //, given by </?(«) = (1234), <^(v) = (13). Then it is clear that h and </>(/,) are rationally 
conjugate for all h G H, and hence H ~ //, in QG. 
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Case 3. The order of H is 6. Since (ZC1) holds for G we must have that H is 
isomorphic to S3. Let H = (u,v) with w 3 = v 2 = 1. Note that Ho has to be trivial, 
otherwise //would be cyclic. Hence the elements of order 2 in H are rationally conjugate 
to (12). Define a monomorphism ip:H— * G by </?(«) = (123) and ip(v) = (12). Then it is 
clear that h and tp(h) are rationally conjugate for all h € H and hence H is conjugate in 
QG to a subgroup of G. 

CASE 4. The order of H is 12. Since Zl\ZG does not have elements of order 6 we 
have, by [2, pp. 134-135], that H is isomorphic to A4. Then the elements of order 2 
are pairwise conjugate in H and case 1 implies that Ho is rationally conjugate to F. If 
<p:H — » A4 is any isomorphism then, clearly, h is rationally conjugate to ip(h) for all 
h € H. Hence, H is rationally conjugate to A4. 

Case 5. H is a group basis. We shall show that H is isomorphic to S4. First note that 
H is solvable. Put H\ = 0 2 >(H). Note that Ho is normal and has order 4 in this case. 
So if H\ is not trivial then H would have an element of order 6 which is, obviously, a 
contradiction. According to case 2, the Sylow 2-subgroups of H are dihedral of order 8. 
Hence, [6, p. 462] implies that H is isomorphic to S4. Denote by <p the extension of any 
isomorphism G - H to the integral group rings. It follows from [19, Theorem 43.6] 
that <p is an inner automorphism induced by a unit of QG. Consequently, H is rationally 
conjugate to G. m 

4. (ZC3) for the binary octahedral group. Let G be the Binary Octahedral group. 
We know that the center Z(G) of G is cyclic of order 2, G/Z(G) ^ S 4 , the Sylow 
2-subgroups of G are generalized quaternion of order 16 and that any group with these 
properties is isomorphic to G (see, for example, [20, 2.1.14]). Moreover, the Fitting 
subgroup F of G is isomorphic to the quaternion group of order8 and G/F ^ S3. Let 
N = Z(G) = (z) and let H>: ZG -» ZG/N be the natural map. 

LEMMA 4. 1 . We can choose a Sylow 2-subgroup P= (a,b : a 8 = 1 , b 1 = a 4 , b~ 1 ab = 
a~ x )o/G and its generators so that T(a) = ( 1 234), F = (a 2 , ab) and a 2 ~ ab in G. 

PROOF. Obviously, we can take a P with ¥(/>) = ((1234), (13)). Since F is the 
inverse image of ((12)(34), (13)(24)), we see that F = {a 2 , ab). Let x 6 G be an element 
of order 3 . Then x~ 1 a 2 x f a 6 . Going down modulo Af to S 4 we see that x~~ 1 a 2 x G {a 2 ** 1 6} 
and consequently a 2 ~ afe in G. ■ 

We also note that a is not conjugate to a 5 in G. For if x~ l ax = a 5 for some x G G then 
x -, f* = P as (a 2 , aft) is the Fitting subgroup. However, N G (P) = P and consequently 
x € P, a contradiction. 

In all that follows in this section we choose P and its generators as in the lemma 
above. If c € G is an element of order 3, then we obtain, looking at S 4 , the following 
representatives of the conjugacy classes of G: 



order of an element 


1 


2 


3 


4 


6 


8 


representatives 


1 


z 


c 


a 2 , b 


zc 


a, a 5 
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We note that ¥ maps the two conjugacy classes of elements of order 4 of G to the two 
classes of elements of order 2 in S4. We begin by proving that the Zassenhaus conjecture 
holds for cyclic subgroups in ZG. We say that a e ZG satisfies the unique trace property 
if there exists a g G G, unique up to conjugacy in G, such that a(g) f 0. 

Proposition 4.2. (ZC1) holds for G. 

Proof. Let a e UiZGbea torsion unit, 0 its image in ZS 4 andg € G. Denote by 
g = V(g). Since (ZC1) holds for S 4 we have that 

(4.3) fid) €{0,1}. 

Note first that the unit group 11\ZG has a unique element of order 2. So we may 
suppose that the order of a is not 2. If o(a) = 3 then we apply Theorem 2.2. If the order 
of or is 6 then we may write a = za 0 , where the order of a 0 is 3 and so we are done by 
the previous case. Going down modulo N we see easily that the only possibilities left for 
the order of a are 4 and 8. 

Let a be a 2-element such that o(a) > 4. We want to show that every element of (a) 
has the unique trace property. Note that z does not belong to the support of a. If g has 
order 3 or 6 then [19, Lemma 38.1 1] implies that a(g) = 0. So we may suppose that g 
is of order 4 or 8. Let g and go be elements of G whose orders are 4 and 8 respectively. 
Going down modulo N it is easy to see that 

(4.4) m = a(g), 

3(go) = a(go) + a(^). 

Since go is not conjugate to g^ in G, there exists an absolutely irreducible character 
X of G so that x(go) ¥ x(go)- lt is easv to see mat me degree of x divides 4 and x is 
not zero on an element of order 8. Moreover, x is faithful as T(a) = ^(a 5 ). Let T be the 
representation associated with x- Then T(z) = -/ and therefore 

(4.5) xteo 5 ) = -xteo). 

We now treat separately the remaining two cases. 

Assume first that a has order 4. It follows from (4.3) and (4.4) that a(go) + a(go 5 ) = 0 
and that there exists a unique, up to conjugacy, elemental G Goforder4suchthatd(gi) ^ 
0. Applying x to a and using (4.3) and (4.5) we obtain that x(«) = xigi) + 2a(go)x(go)- 
It follows from the equalities g\ = z = or 2 that the eigenvalues of T(a) and Y{g\ ) are ±i. 
Note that in G every element is conjugate to its inverse so x is real-valued. Consequently, 
X(cr) = x(g\ ) = 0 and so a(go) = 0. Thus any element of (a) has the unique trace property 
and in view of [ 1 9, Lemma 4 1 .5] a ~ g\ in QG. 

Finally assume that o(a) = 8. By the same reasoning we obtain that a(g) = 0 if 
o(g) f 8 and a(g 0 ) + d(g^) = 1 . Hence, 



(4.6) 



X(«) - [a(go) - a(d)]x(go) = [2d(g 0 ) - l]x(go). 
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The equalities or = z = g\ imply that the eigenvalues of or and go are primitive roots 
of unity of degree 8. Since \ is real-valued and x(go) / Owe see easily that the only 
possibilities for \(a) and x(go) are ±y/2 and ±2\/2. Using this fact and (4.6) we obtain 
that 2d(go) — 1 = ±1 and so a(go) is 0 or 1 . It follows from the former case, that every 
element of (a) has the unique trace property and so, by [19, Lemma 41 .5], either or ~ go 
or a ~ in QG. * 

THEOREM 4.7. G satisfies (ZC3). 

PROOF. As we already mentioned 11\ ~LG has a unique element z of order 2, which is 
central, and we denoted N = (z). So if H is a finite non-cyclic subgroup of ZI\1G then 
the Sylow 2-subgroups of H are either cyclic, or quaternion of order 8 or generalized 
quaternion of order 16. Moreover, since (ZC3) holds for S4 and this group does not have 
subgroups of order 6, Zl\ ZG does not contain subgroups of order 12. 

Let \H\ = 8. Suppose first that H < 1l{ 1 + A(G, F)). Then, by (ZC1), any 1 f h G H 
is conjugate in QG to a 2 ~ ab. Therefore, if </>://—> F is any isomorphism, h is rationally 
conjugate to <p{h) for all AG//, and Lemma 2.6 implies that H and F are conjugate in 
QG. 

If H is not contained in tl( 1 + A(G, F)) then it is easily seen that, going modulo N, 
we may choose generators h 0 , h\ of H such that h 0 ~ b and h\ ~ a 2 in QG. We now 
define a homomorphism <p:H —> (a 2 ,b) by <^(/» 0 ) = 6, <p(A0 = a 2 . Since ¥(Mo) £ 
1 + A(S 4 , Fit(5 4 ))) it follows that Mo ~ A 2 * in QG and Aj/» 0 = zh { h Q ~ za 2 6 = a 6 fc 
in QG. Hence h and tp(h) are rationally conjugate for all h G // and consequently so are 
//and(fl 2 ,6). 

Suppose now that the order of //is 16. We have that H ^ P. Choose generators w, 
v for H so that - (1234) and T(v) ~ (13) in Q5 4 . It follows, by proposition 4.2, 
that v ~ b in QG and either u ~ a or u ~ a 5 in QG. In the later case we consider 
a 5 instead of a, so we may suppose that u ~ a. Define an isomorphism //—»/* by 
V?(«) = a, </?(v) = 6. Observe that H^t/v) is rationally conjugate to (1234)*(13). So if k 
is even then ^(i/v) ~ (24) ~ (13) and consequently i/v ~ b in QG. If A is odd, then 
¥(«* v) ~ (14X23) in QS4 and, hence i/v ~ a 2 ~ a£ in QG. So we proved that A ~ <p(h) 
for all A G // and, therefore, H and P are rationally conjugate. 

Let \H\ = 24. Since S 4 satisfies (ZC3) it follows that ¥(//) ~ A 4 in Q5 4 . Since /i 4 has 
a normal Sylow 2-subgroup it follows that H also has a normal Sylow 2-subgroup Hq. 
Hence // = // 0 x (v) with v 3 = 1. Clearly // 0 is the quaternion group of order 8 and as 
T(// 0 ) ~ T(F) in Q5 4 , going down modulo F, it is easily seen that H 0 < 1l( 1 + A(G, F)) . 
Consequently, H 0 is rationally conjugate to F. Let c G G be an clement of order 3, 

Gi = F x (c) and y.H -4 G\ any isomorphism. Recall that the conjugacy classes of 
elements of order 3 and 6 are respectively represented by zc and c. From this it easily 
follows that </?(A) ~ h in QG for every h € H and hence // and G| are rationally 
conjugate. 

Finally let \H\ = 48. It follows from the information above that HjZAH) - 5 4 and the 
Sylow 2-subgroups of H are isomorphic to P. Hence, H must be the Binary Octahedral 
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Group. Let </?://—► G be any isomorphism. Theorem 3. 1 and Proposition 4.2 imply that 
ip(h) ~ h in QG for every h G H with o(A) ^ 8. Let o(/j) = 8 and suppose that <p(h) is not 
rationally conjugate to h. We have that G = {P, c), c 3 = 1 and G\ = F xi (c) has index 2 
in G. Define a map 6 by a — ► a 5 and g — > g for any g 6 G\ . Since the elements of G\ are 
fixed by this map it follows that it is an automorphism of G. It is easy to check now that 
if we replace tp by <p0, we get <p(h) ~ h in QG for all h 6 H and consequently A/ and G 
are rationally conjugate. ■ 

5. (p-ZC3) for some solvable groups. 

THEOREM 5.1. Let G be a solvable group such that any Sylow subgroup of G is either 
abelian or generalized quaternion. Then G satisfies (p-ZC3). 

Proof. Let H be a finite p-subgroup of ll\ ZG. In view of Proposition 2. 1 1 we may 
assume that p = 2 and the Sylow 2-subgroups of G are generalized quaternion. If the 
Fitting subgroup F of G is not a 2-group, then G contains a non-trivial normal subgroup 
N of odd order. Since the factor group G/N satisfies the assumption of the theorem we 
use Theorem 2.2 and induction on |<7|. 

Let F be a 2-group. Since G is solvable, C G (F) = Z(F) [1 8, p. 144] and, consequently, 
GjZAF) = N G (F)/C G (F) is a subgroup of Aut(F). According to [17, Proposition 9.10] 
if F is not isomorphic to Q%, the quaternion group of order 8, then Aut(F) is a 2-group 
and the result follows from [22]. Let F * Q%. Then Aut(F) * S 4 , \Z(F)\ = 2 and, hence, 
|G| divides 48. By [22] we may suppose that G is not nilpotent. If |G| = 24 then G has a 
normal Sylow 2-subgroup and we can use Theorem 2.9. If |G| = 48 then G is the Binary 
Octahedral group. In this case we apply Theorem 4.7. ■ 

COROLLARY 5.2. A finite solvable Frobenius group satisfies (p-ZC3). 

PROOF. By [18, 10.5.6]G = Wx^whereWis nilpotent, (\N\,\X\) = 1 and the Sylow 
/7-subgroups of * are either abelian or generalized quaternion. Hence, the result follows 
from Corollary 2.4 and Theorem 5.1. ■ 

THEOREM 5.3. Let Gbea finite solvable group and L = L(G) the last non-trivial term 
of the lower central series of G If p 4 does not divide \G\ for any prime p dividing \L\, 
then G satisfies (p-ZC3). 

Proof. Let H be a finite /^-subgroup of 1l\ ZG. If p does not divide \L\ then, since 
G/L is nilpotent, we apply Theorem 2.2 and the theorem of Weiss [22]. 

Let p divide \L\ and let F be the Fitting subgroup of G. If F is not a p-group, then 
N = Opi(F) is a non-identity normal subgroup of G. It is easy to see that the factor group 
G/N satisfies the hypothesis of the theorem, so we may use Theorem 2.2 and induction 
on the order of G. 

Let F be a p-group and P a Sylow p-subgroup of G. In view of Proposition 2. 1 1 and 
[19, Theorem 41.12] we may assume that P is not abelian and not normal in G. In fact 
l^l = f? because p 4 does not divide \G\. Now the same arguments as in [7, pp. 4908-4909] 
shows that p = 2 and G - S 4 . Thus, the result follows from Theorem 3.1. ■ 



Copyrighted material 



FINITE SUBGROUPS IN INTEGRAL GROUP RINGS 



1179 



REMARK. The proof of the theorem shows that if H C ( U\ ZG is a finite subgroup 
whose order is relatively prime to that of L then H is rationally conjugate to a subgroup 
ofG. 
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MARKOV'S THEOREM FOR ORTHOGONAL 
MATRIX POLYNOMIALS 



ANTONIO J. DURAN 



Abstract. Markov's Theorem shows asymptotic behavior of the ratio between the 
n-th orthonormal polynomial with respect to a positive measure and the n-th polynomial 
of the second kind. In this paper we extend Markov's Theorem for orthogonal matrix 
polynomials. 



1. Introduction. A. Markov established in 1895 (see [M]) the following result, 
which is now known as Markov's theorem: 



where n is a probability measure on the finite interval [a,b], (p n ) n is the sequence of 
orthonormal polynomials with respect to \i and (q„)„ is the corresponding sequence of 
polynomials of the second kind, defined by 



The hypothesis \i having compact support is too restrictive, and actually the determi- 
nacy of the measure n is a sufficient condition. Even for some families of indeterminate 
measures Markov's theorem holds (see the recent survey about Markov's theorem [B]). 

The purpose of this paper is to extend Markov's theorem for orthogonal matrix poly- 



We consider aNxN positive definite matrix of measures W (for any Borel set A C R, 
W{A) is a positive semidefinite numerical matrix), having moments of every order, i.e., 
the matrix integral 



exists for any nonnegative integer n. 

The matrix inner product defined in the usual way by W in the space of matrix poly- 
nomials has associated a sequence of orthonormal matrix polynomials (P„)„, satisfying 



P„(t) is a matrix polynomial of degree n, with a non-singular leading coefficient and is 
defined upon a multiplication on the left by a unitary matrix. 
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nomials. 




Jp„(t)dW(t)P* m (t) = 6 n/ „I, n,m>0. 
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As in the scalar case, the sequence of orthonormal matrix polynomials (P„)„ satisfies 
a three-term recurrence relation 



where A„ are non-singular matrices and B n are hermitian. Without loss of generality, we 
assume P 0 (t) = /. (Here and in the rest of this paper, we write 9 for the null matrix, the 
dimension of which can be determined from the context.) 

This three term recurrence relation characterizes the orthonormal ity of a sequence 
of matrix polynomials with respect to a positive definite matrix of measures (see, for 
instance, [AN] or [DL]). In [Dl], [D2] and [DV] a very close relationship between or- 
thogonal matrix polynomials and scalar polynomials satisfying a higher order recurrence 
relation has been established. 

The corresponding matrix polynomials of the second kind are defined by 



We say that the positive definite matrix of measures W is determinate if no other positive 
definite matrix of measures has the same moments as those of W, i.e. , the positive definite 
matrix of measures W is uniquely determined by the moments J f dW(f) (n € N). 

To establish Markov's theorem, we need the following definitions: A„ stands for the 
set of zeros of the matrix polynomial P„, i.e., the zeros of det(P„). In [DL], it is proved 
that these zeros are real and have multiplicity at most N. We finally put 



It is proved in [DL] that orthogonal i/ing matrix of measures \i for the matrix polynomials 
(P„)„ can be found as weak accumulation points of a sequence of discrete measures /x„ 
with support precisely A„. Therefore, for a determinate matrix of measures W, we have 
supp(0O C r. 

The main result of this paper is the following matrix extension of Markov's Theorem. 
THEOREM 1.1. Assume that W is determinate. Then 



and the convergence is uniform for z in compact subsets ofC\T. 

We will show that in the matrix version of Markov's theorem, the matrices P~ x and 
Q n must be multiplied in the order P„ x Q n , otherwise the result could be false. 

To prove Markov's theorem we find a quadrature formula (Section 3) for the inner 
product defined by the matrix of measures W in the space of matrix polynomials. The 
matrix coefficients, which appear in that quadrature formula, will be given in a closed 
expression, and this expression will be the key to establish the matrix version of Markov's 
Theorem (see [SV, and DL] for other versions of the quadrature formula). 



tP n (t) = A n+[ P n+[ (t) + B n P n (t) + A;P n ^(t), n > 0, />_,(/) = 0, 




(l.D 



r = n N > 0 M N , where M N = U n > N A n . 
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In the proof of this quadrature formula we use a property of the zeros (Section 2) of 
the n-th orthonormal matrix polynomial P n which will play a fundamental role in the 
whole theory of orthogonal matrix polynomials: 

If a is a zero of P„(t) of multiplicity/?, then rank(P n (a)) = N-p (hence p < AO, 

and the matrices (AdjfP,,^)))*'' %) and P*„_ x (a)A n define two isomorphisms 
from the space of right eigenvectors of P n (a) associated to the eigenvalue 0 into 
the space of left eigenvectors of P n (ct) associated to the eigenvalue 0. The inverse 
mapping of P* n _\{a)A n is that defined by the matrix Q„(a). 

For a given matrix A, we denote by Adj(/I) the classical adjoint, i.e., the matrix 
uniquely defined by the property 

A Adj(/4) = Adj(A)A = de\(A)I. 

That the zeros of orthogonal polynomials are simple (in the scalar case) is a well-known 
property, which is not true in the matrix case. Precisely, the property we have noted above 
will play in the matrix case the same role as that of the non-multiplicity of the zeros in 
the scalar case. 

To complete this paper, we give a generalization of Markov's theorem showing the 
asymptotic behavior between the sequence of orthonormal matrix polynomials and the 
sequence of k-th associated polynomials (Section 5) 

2. Zeros. We start with the following lemma which contains the matrix version of 
some classical formulae for orthonormal scalar polynomials. The proofs work as in the 
scalar case and so are omitted. 

Lemma 2. 1 . (I) The Christoffel-Darboux formula and some special cases 

P* H _ y (z)A n P n (w) - OzM;/>„_,(w) 



(2.1) 



= (w-z)£^(z)/>*(vf), ZjW ec. 



(2.2) P*„_ { (z)A n P n (z) - nWlPn-xiz) = 6, zeC. 

(2.3) c 1 (z^( Z )-p;(2K^,W=EWK4 zee. 

(2) Some particular cases of the Green formula: 

(2.4) •=! 

= l + {w-z)Y,Pl(z)Qk(*), z,w€C. 



(2.5) K-lWnQn(z) - PtizWQn iiz) = I, Z € C. 
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(3) The Liouville-Ostrogadski formula 

(2.6) OAKUti - = 4T 1 - 

To prove the remarkable property about the zeros of P„ stressed in the introduction, 
and the quadrature formula, we need the following lemma: 

Lemma 2.2. Let A(t) be a N x N matrix polynomial and let a be a zero of A(t) of 
multiplicity p, i.e., a zero of multiplicity p of the scalar polynomial det(^(/)). We put 

L(a,A) = {v e C N : vA(a) = 9}, R(a,A) = {v € C* : A(a)v* = 9}. 

Ifdim(L(a,A)) = dim(R(a,A)) = p. then {hd}(A{tjj) (t \a) = 9, for I = 0, . . . ,p - 2 
and ^Adj(/1(0)) (/ ' (a) ^ 0. Moreover rank^Adj^CO))^ V) = p and 

(Adj(^(0)) (P_1) (a) 

defines a linear mappingfrom C N onto L(a,A) which is an isomorphism from R(a,A) into 
L(a,A). 

Proof. It follows straightforward from Lemma 2.2 of [DL] that 
(Adj(^(/))) (/) (a) = 9, for / = 0, . . . ,p - 2. 

We now prove that ^Adj(,4(/)) j (a) ^ 9. Since a is a zero of A(t) of multiplicity p, 

by differentiating the formula Adj\A(t))A(t) = dot .-!(/)/ and taking into account what 
we have already proved, we obtain that 

(2.7) ^Adj(A(t))Y~ l \a)A(a) = 9, and 

(2.8) (Adj(^(/)))°' ) (a^(a) + (Adj^O))^' W(«) = (det^W 

If (Adj(^(/))) 0 ' (a) = 9, (2.8) gives that 

(Adj(/!(/))) W <aX(«) = (det^O^W- 

But this is impossible because A(a) is singular but (det^(/)) (p) (a)/ is non-singular {a is 
a zero of multiplicity just p of det(/l(0)). 
The formula (2.7) gives that the matrix 

(Adj^^'V) 
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defines a linear mapping 

[A^(A(t)))^\a):C N ^ L{a f A) 
v^v(Adj(^(/))) (/ "' ) (fl). 
For v € R(a,A), v ^ 0, (2.8) gives that 

v{Adi(A(t))Y~ n (a)A'(ay f 9. 

So, the linear mapping defined by ( Adj(^(/)) (a) is injective in /?(a,/i). Hence, 

rank ( Adj (/!(/)) ^V) > dim(/?(a„4)) = p. 

But (2.7) gives that the rows of (Adj^/)))^ (a) are vectors of L(a,A), hence 

rank(Adj(^(/))) <P " l) (a) < dim(L(a,A)) = P- 

And the proof is finished. ■ 
Some properties of zeros of orthogonal matrix polynomials on the real line have been 
established recently in [SV] and [DL] (see also [Z]). We complete here those results by 
proving the property we noted in the introduction of this paper. 

Theorem 2.3. If a is a zero of P„(t) of multiplicity p, then 

(1) a is real, p < N, rank(/ J „(a)) =N-p, and dim(R(a,P„)) = dim(L(a,P n j) = p 
(R(a, P n ) and L(a, P„) are, respectively, the spaces of right and left eigenvectors 
of P„(a) associated to the eigenvalue 0). 

(2) The matrix P*_ , (a)A n defines an isomorphism from R(a, P„) into L(a, P n ). Its in- 
verse mapping is the isomorphism defined by the matrix Q n {a). 

(3) The matrix ( Adj ( />„(/)) \ (a) defines a linear mapping from C N onto L(a, P„) 
which is an isomorphism from R(a, P„) into L{a, P„). 

(4) [M]{P n {t))f y \d)P n {a) = P n (a)[Adj(P n (t))Y' l \a) = 0 and 

mnk(Adi(P n (t))) {P ~ i \a)=p, 

being the p linearly independent rows o/^Adj(P„(/))) <P l \a) a basis of the linear 
space of left eigenvectors ofP„(a) associated to 0. 

PROOF. ( 1 ) It is contained in parts ( 1 ) and (2) of Theorem 1 . 1 in [DL]. 

(2) First of all, we prove that if v e R(a,P n ) then vP* n ^(a)A n € L(a,P n ). Indeed, if 
P„(a)v* = 0, the formula (2.2) gives that vP* n _ x {a)A n P n {a) = 0, that is, vP* n _ x {a)A n G 
L(a,P„). 
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For a vector v € R(a,P n \ the formula (2.5) gives 
(2.9) vP* n _ x {a)A n Q n {a) = v, 

and so, vP* n _ , (a)A„- \ ^ 0. This proves that the mapping is injective. Since the dimension 
of R(a, P„) and L(a, P„) is the same, the mapping is automatically an isomorphism. 

(2.9) shows that its inverse mapping is just that defined by Q„(a). 

(3) and (4) From (1) we have that dim(R(a,P„j) = dim(Z,(a,/ > „)) = p. Then (3) and 
(4) of Theorem 2.3 are straightforwardly deduced from Lemma 2.2. ■ 

Theorem 2.3 can be extended for perturbations of the orthonormal polynomials. In- 
deed, let A be a matrix satisfying that A n A is hermitian, and let us consider the polynomial 
P n (t) - AP n -x(t). Then, Theorem 2.3 holds for this polynomial, i.e.: 

Theorem 2.4. If a is a zero of P n {t)-AP n -\{t) of multiplicity p, where A n A = A*A*, 
then 

(1) aisreal.p < N, rank(/ , „(a)-^/ > n _,(a)) = N-p, andd\m(R{a,P n -AP n - X j) = 

dim(L(a,P n -AP n ^))=p. 

(2) The matrix P* n _ { (a)A n defines an isomorphism from R(a,P n — AP„-\) into 
L(a,P„ — AP„-\). Its inverse mapping is the isomorphism defined by the matrix 
Q n (a)-AQ n ^(a). 

(3) The matrix ^Adj (/>„(/) — AP„-\(t)) j (a) defines a linear mapping from C N 
onto L{a,P„ — AP n -\) which is an isomorphism from R(a,P„ — AP„-\) into 
L{a,P n -AP n . { ). 

(4) i} 

(Adj(P,(/) -AP^t(t)))^\)(PM -AP n ^{a)) 

= (P„(a)-^P„_ 1 (a))(Adj(P fl (0-^P„_ 1 (0)) (P_,) (a) = 0 

and 

rank(Adj(/>„(/) - AP^t)))^^) = p, 

being the p linearly independent rows of ( Adj (P„(t) — AP n - i (0) ) (o)a basis 
of the linear space of left eigenvectors of P n (a) — AP n -\(a) associated to 0. 

By modifying in an appropriate way the formulae showed in Lemma 2. 1 (this can be 
easily done by using that the matrix A„A is hermitian), the proof of Theorem 2.4 exactly 
works as that of Theorem 2.3. 

3. The quadrature formula revisited. Quadrature formulae have been found re- 
cently for orthonormal matrix polynomials by Sinap and Van Assche (see [SV]) and 
Duran and Lopez-Rodriguez (see [DL]). Here and using other different approach, we 
improve these quadrature formulae by giving a closed expression for the matrix coeffi- 
cients which appear in the formula. This expression will be the key to prove the matrix 
version of Markov's Theorem. 



Copyrighted material 



1186 A. J. DURAN 

THEOREM 3. 1 . Let n be a nonnegative integer. We write x nJc (k = 1 , . . . , m) for the 
different zeros of the matrix polynomial P„ ordered in increasing size (hence m < nN) 
and V nJl for the matrices 

(3.1) r„,*=- 5-^ (Adj(P n (t))) (li ~'\x nJc )Q n (x nJc ), *=l,...,m, 

(det(/>„(/))) (x n<k ) 

where /* is the multiplicity of x^. 

(1) For any polynomial P with dgr(P) < In — 1 the following formula holds 

(3.2) fp(t)dW(t)=Ep(x nJ[ )r n j t . 

J k=\ 

(2) The matrices V nJc are positive semidefinite matrices of rank k,k= 1 , . . . , m. 

It is worth noting that the smaller the multiplicity of a zero \ r ,i is, the bigger the 
singularity of the matrix T nJ( , is. For instance, if x„jt is simple, then r n<k has rank one. 

Proof. To prove ( 1 ) of Theorem 3 . 1 we use the following surprising fact: 

Although the zeros of P„ (i.e., the zeros of det(P n )) can be of multiplicity bigger 
than one (at most AO the decomposition 



(3.3) P(z)/>„ l (z)=£-%- ifdgr/></,^l, 

k=l z-x nM 

is always possible. 

Part (4) of Theorem 2.3 and Lemma 2.2 show the reason for this remarkable property: 
indeed, if a is a zero of multiplicity p for P„ then 

Adj(P B (a)) = (Adj(/>„(/)))'(a) = • • • = (Adj (/>„(/)) ^(a) = 6, and 

(Adj^w)) 0 " 0 ^) ^ e. 

This means that a is a zero of multiplicity at least/7— 1 of each entry of Adj (P„(z)) . Hence 
a is a zero of multiplicity at least p — 1 of P(z) Adj(/ , „(z)) for any matrix polynomial 
P, and so, if dgr(/>) < n - 1, (i.e., dgr(P Adj(/>„)) < nN - 1 < dgr(det(/>„))), the 
expression (3.3) holds for 

. P(z)Adj(P n (zj) 
P(z)P~ n \z)= K '/ p \' ,} . 

det(/yz)) 

We now prove (1) of Theorem 3. 1 . Let P be a matrix polynomial of degree less than 
or equal to In - 1. Since P„ is a polynomial with non-singular leading coefficient, we 
write ([G] p. 78) 



m 



(3.4) P(t) = C(t)P n (t) + R(t), 
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where C(t) and /?(/) are matrix polynomials with dgr(/?) < n — 1 . 
Then we have 

(3.5) P(t)P;\t) = C(t) + R(t)P- [ (tl 
when / is not a zero of P„. Since dgr (/?(/)) <n - 1 we can write 

*(0P„-'(0 = EC,a— !— , 
*=i t-x n ,k 

where the matrices C„,* are given by 

c »* = 1 ko3 ^)(Adj(P„(0)) (/j_1) (^). 

(det(/>„(/))) &*) 

Since P„(x nJ( )l Adi(P„(t)) J (*„,*) = 0 (see part (4) of Theorem 2.3), (3.4) gives that 
the matrices C„jt can be written in the following way 

(3. 6) C nJc = ^^(Adj^O))*"' W). 

(det(P n (0)j (x nJc ) 

Then (3.5) gives 

m p ( t \ 

p(0 = c(o/ > w (/) + E^-^ i -. 

From (3.6) and part (4) of Theorem 2.3, we have 

Pit) = c(t)P n (t) + ± c K /* {t) - PmM . 

k=i t-x„, k 



From the definition of the polynomial Q„ it follows that 

m 

E 

*=1 



/ P(t)dW(t) = I C(t)P n (t)dW(t) + £ C Hjc Q n (x Hjt ). 

J J I I 



But dgr(Q < n, so the orthonormality of P„ gives that / C(t)P„(t) dW(i) = 6. Hence, we 
have 

JmdW(t) = ±C nJ[ Q n (x ttJt ). 

Then, part (1) of Theorem 3.1 follows from the definition of the matrices r nJ( and C nJc 
(see (3.1) and (3.6)). 

We now prove part (2) of Theorem 3.1: 

We proceed in several steps: 

Step 1 . The matrices r nJl (k= 1 , . . . , m) are hermitian. 
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Proof. If we prove that the matrix P n 1 (t)Q„(t) is hermitian when t G R is not a zero 
of P„, Step 1 would follow from the decomposition 

k=\ t-XnJc 

The orthogonality of P n with respect to any other polynomial of degree less than n gives 
that 

P~ n \t)Qn(t) = J K W™Z* ^ 

= J p- n l (t) Pn(t ]z P x n{x) dw(x){m - nwynr'it) 

= J p- n \t)(p n (t) - p n (x)) dW{x) n{t \~J^ {x \ Kr\t) 

J l — X 

that is, P~ l (t)Q»(t) is hermitian when t G R is not a zero of P„. ■ 
Step 2. rank(r n> ) = l k ,k= \,...,m. 

Proof. Theorem 2.3 gives that rankf Adj(f n (/)) J (x nfk ) - l k . Since the left 

eigenvectors of (Adj (/»„(/)) J (*„,*) associated to 0 are left eigenvectors of r n< * asso- 
ciated to 0 (see (3. 1)), it will be enough to prove that left eigenvectors of r„ k associated 

to 0 are left eigenvectors of ( Adj(P n (0) 1 (jc„^) associated to 0. 

Let u be for which v = w (Adj (/>„(/)) ) ( '' V»a) ^ 0. Part (3) of Theorem 2.3 shows 
that v G L(x ntk ,P„). Then the formula (2.6) gives 

Since the matrix A~ l is non-singular, we have that vQ n {x nJ ,) ^ 9, and taking into account 
the definition of v and r n> , we obtain uT nJl ^ 9. And Step 2 is proved. ■ 
We take vectors v u ...,v, k satisfying the following properties: 

(1) v, is a left eigenvector of the matrix r n<k associated to an eigenvalue or, ^ 0, 

(2) vrf = Sjj, ij = 1,...,/*. 

Step 3. For / = 1,. ..,/*, constants 0, ^ 0 exist such that 

V ( (Adj(P„(0)) (/j_1) Ct^) = frvft-xiXnjMn. 

Proof. From part (3) of Theorem 2.3, we have that v,( Adj (P n (t))J (x njc ) G 
L(x nJc ,P„). Since the matrix P* H _ x (x nJk )A n defines an isomorphism from R(x nyk ,P„) into 
L(x nJt , P n ) (Theorem 2.3(2)), we have that 

(3- 7) v ( (Adj(P„(0)) ( '*~ V*) = «*/^ifctfH,, 
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for certain u, £ R(x nyky P n ). Since Q„(x nJl ) is the inverse mapping of P* n _ x (x nJc )A ny we 
have from (3.7) that 

v,(Adj(P ll (0)) (/ *" l) (x lll *)a(x M ) - u t . 

Then, from the definition of the matrix T nJl (see (3.1 )), we deduce that 

j_ 

(dctP n (t)) { ' l \x nJc ) 



Since ViT nJl = or,v„ it is enough to take /?,■ = Qr, (detP B (/)) (/ * ) (x rt j t ). ■ 
Finally, we prove that v,T„^v* > 0, i.e., the matrix r n> * is positive semidefinite. In 
Step 3, we have proved that v, = jiij, for certain u, € /?(*/,,*, P*), then we deduce that 
v/ G R(x nt k,P„). From the formula 

(Adj(p„(0)) (/4) (x flJk )P /l (^) + (^(iUOj^Wyi^) 

= (detP w (/)) (W (x M )/, 

and Step 3, we obtain 

(3. 8) flfl^ifetfM^C^M = (det/»„(0) (/i) (JC^). 

Step 3, (3.8), (2.3) and (2.5) give that 

v,(Adj(/> n (^))) (/4_1) ^(^K 

v ' r ^ v <* ■ — r: 

_ PiViP* n -l(XnJt)A ll Q n (x l ,j t X 
PiViPl^jMnP'niXntX 

_ Vi{P:- X {x n jMnQ n {x nJl ) - P* n (x nJc )A* n Q n ^x n , k ))v; 

Vi{n^{x nJ MnP' n {x nJl ) - PXXnjWP^*))^ 



1 



And the theorem is proved. ■ 
Theorem 3.1 can also be extended for perturbations of the orthonormal polynomi- 
als. Indeed, let A be a matrix satisfying that A„A is hermitian, and let us consider the 
polynomial P n {t) - AP„- X {t). Then, Theorem 3. 1 holds for this polynomial, i.e.: 

THEOREM 3.2. Let n be a nonnegative integer. We write x nJ( (k = I,..., m)for the 
different zeros of the matrix polynomial P„ - AP„-\ (where A„A = A*A*J ordered in 
increasing size (hence m < nN) and r nJc for the matrices 

r„,* = ! -jjrj— (Adj(p n (/)-^ n _,(/))) (/4_, W) 

(det(P n (t) - AP n ^(t))j (x nJl ) 
■ (QniXnJc) ~ AQ n - X {x nJ( j), k=\,...,m, 
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where /* is the multiplicity of x„jt. 

(1) For any polynomial P with dgrfP) < In - 2 the following formula holds 

(3.9) fP{t)dW{t)=±P{x nJ ,)Y nJl . 

J k=\ 

(2) The matrices are positive semidefinite matrices of rank for k — 1 , . . . , m. 

Using Theorem 2.4 instead of Theorem 2.3 the proof of Theorem 3.2 exactly works as 
that of Theorem 3. 1 . It is worth noting that (3.9) can not be extended for dgr(P) < In - 1 
because of 

jc{t)(P n (t) ~ AP„^(t)) dW{t) = 9 

can only be guaranteed when dgr(C) < n - 1 . 

Remark 3.3. In the scalar case, the coefficients of the quadrature formula satisfy a 
remarkable property called Markov-Stieltjes inequalities: 

>^nj < / " dii(x) < 2Z X„j, k = 1, . . . ,n, 

Jml j=\ 

where x„ t \, . . . ,x n<n are the zeros of the orthonormal polynomial p„, and are the 
quadrature weights given by 

This property was conjectured by Chebyshev in 1874, and proved independently by 
Markov and Stieltjes ten years later. We present here a counterexample showing that 
these Markov-Stieltjes inequalities are no longer true for orthogonal matrix polynomi- 
als. The reason is the highly singular character of quadrature weights in the matrix case. 

Indeed, let p\ , P2, (p\jt)n and (p2, n )n be two positive measures and their sequences of 
orthonormal polynomials. We define the matrix of measures W by 

*-(: a- 

Then it is clear that its sequence of orthonormal matrix polynomials is 

We assume that the measures p\ and \ii have the same support [a, b] and that p\, n and pi^ 
interlace their zeros. Then an easy computation gives that the coefficients in the matrix 
quadrature formula are 




for*= l,3,--.2»-l, 
for A- = 2,4,--,2w, 
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where A liB< *, A 2 , n ^, (k = 1, ... ,n) are the quadrature weights for the measures m , n 2 
respectively. Then, since 

r *-[ 0 oj' 

is a singular matrix, it is clear that 

dw{x) ^r„,,, 

because of the non-singularity of J? 1 *' dW{x). 

4. Markov's theorem for orthogonal matrix polynomials. We are now ready to 
prove the matrix version of Markov's theorem 

Proof of Theorem 1.1. First of all, we prove that 

dW{t) 



for z G C \ T (see ( 1 . 1 ) for the definition of T). 

We consider the sequence of discrete matrices of measures (ji n )„ defined by 

^ = E^r nJt , n>0, 

k=i 

where x n<k (k = 1, ... ,m) are the different zeros of the matrix polynomial P„ ordered 
in increasing size, and r n>k are the positive semidefinite matrices defined by (3.1) and 
which appear in the quadrature formula (3.2). 

From the definition of these matrices (see (3.1)), it follows straightforwardly that 

<4. i) fj'tt&M = ± r«-±- = J 

if z is not a zero of P„. Taking into account (4.1), it will be enough to prove that 

dfi n (t) _ r dW(t) 



n—ooJ Z — t J Z — t 



for z G C \ T. If not, we find a complex number z G C \ T, an increasing sequence of 
nonnegative integers (n m ) m and a positive constant C for which 

where we write || • || 2 for the spectral norm of a matrix. 

If we proceed as in Section 2 of [DL], taking two increasing sequences (a k ) k , (b k ) k for 
which a k , b k — > +oo, we can obtain (by using Banach-Alaoglu's theorem) a subsequence 
(l m ) m from (n m ) m , and a positive definite matrix of measures v such that for k > 0 

(4. 3) Jta fcm*H<® = /V(')<M0, 



Copyrighted materia 



1192 



A. J. DURAN 



for any continuous matrix function / defined in [— Now it is easy to prove (see 
Section 2 of [DL] for more details) that the k-th moment of the matrix of measures u is the 
limit of the £-th moment of the matrices of measures \i\ n . But the quadrature formula (3 .2) 
gives that the &-th moment of the matrix of measures (k < 2l m — I) is precisely the 
k-th moment of the matrix of measures W . Since W is a determinate matrix of measures, 
we conclude that v = W. Then (4.3) gives that for k > 0 

(4. 4) lim /* /(/) dp,.® = f bk f(t) dW(t). 

m — »oo J— an J —Ok 

By taking k and then l m big enough, from (4.2) and (4.4) we obtain 

We write So for the first moment of the matrices of measures which is the first moment 
of the matrix of measures W. Then we have from (4.5) and the definition of the spectral 
norm that 

^ max (^i-ir^|)||/»'""-«| 2 

But this implies C = 0, and therefore, (4.2) is not possible. 

We now prove that the analytic functions which form the entries of the matrix J 
are uniformly bounded in compact sets of C \ T. Then, the uniform convergence in com- 
pact set of C \ T will follow from Stieltjes-Vitali's theorem. 

Given a compact K c C \ T, we notice that K n M N = 0, for N sufficiently big, and 
then there exists C> 0 such that 



1 



<C forzEK and / G My. 



z-t 

Then, for « > N and v G the positive definiteness of n„ gives that 

| v/ ^ v .| =/ «l< c/v(/M()v . = vSoV<i 

and now it is easy to finish. ■ 
It is worth to note that the order in which the polynomials P~ 1 and Q„ appear multi- 
plied in the matrix version of Markov's theorem, i.e., P~ l (z)Q„(z), is essential to guar- 
entee the validity of this result. Indeed, let W be a positive definite matrix of measures 
and (/>„)«, {Q n )n its orthonormal matrix polynomials and polynomials of the second kind, 
respectively. Let us consider a non-singular matrix C, and the positive definite matrix of 
measures defined by R = CWC*. It is clear that {P„Cr ] )„ and (Q„C*) are, respectively, 
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the sequence of orthonormal matrix polynomials and the sequence of polynomials of the 
second kind for R. Hence, if Markov's theorem held for Q n {z)P~\z), we would have 

dW(t) 



WmQ n {z)CCP- n \z)= C I 

n J Z — I 



which, in general, is clearly false. 

Markov's theorem can be extended for perturbations of the orthonormal polynomi- 
als: Indeed, let A be a matrix satisfying that A„A is hermitian, and let us consider the 
polynomial P„(t) - AP n _ | (/). We write A„j for the set of zeros of the matrix polynomial 
P B -^P„_,,and 



Fa = (^h>oM N j, where M NtA = \J„>N& nr A- 

Then 

THEOREM 4. 1 . Assume that W is determinate and that A„A = A*A*„for n>0. Then 

\im (P n (z)-AP n ^(z))~ l (Q n (z)-AQ n Mz)) = / ^37 forze C \ T, 

and the convergence is uniform for z in compact subsets of C\r A . 

Using Theorem 3.2 instead of Theorem 3.1 the proof of Theorem 4.1 exactly works 
as that of Theorem 1.1. 

5. The *-th associated polynomials. For k > 1, the *-th associated polynomials 
(fW) n are defined by the formula 

(5. 1) P%\x) = / Pn + k(x) x ~ ^* (/) dW{i)P* k _ x {t\ n > 0. 

The orthogonality of the polynomials (P n )„ shows that the degree of fyl is just n. 

Associated polynomials already appear in Stieltjes' fundamental work [S]. (See the 
survey [V] for more details). 

These *-th associated polynomials satisfy the following two recurrence formulae 

(5.2) */*!(*) = ^ HI /*^ *>0, 
and 

(5.3) x^\x) m ^l\\x)Al+P^\x)BU + ^.'^M*-. 

with initial condition A 0 = /, /*_*] (jc) = 6 and P$\x) = A~ k x . Indeed, we have 

-j^^dwm^t), 
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but also 

Formulae (5.2) and (5.3) now follow from the three term recurrence formula for (P „)„ 
and the orthogonality of these polynomials. 

From the formula (5.3) and the three term recurrence formula for (P n )n one obtains 
the following quasi Christoffel-Darboux formula 

Theorem 5.1. The following formulae are valid: 

(x-t)± = P„(x) - />„(/). 

and 

*=1 



Proof. Indeed, it is enough to write 

and compute the matrix coefficients by using the orthonormality of (P n )n- ■ 
The following generalization of Markov's Theorem is a consequence of Theorem 1.1, 
the formula (5.3) and the three term recurrence formula for (/>„)„ (see [V] for the scalar 
version): 

THEOREM 5.2. Assume that W is determinate. Then for k > 1 

Urn P- n \z^\ k {2) = / dW(t)^=^ forz e C \ T, 

and the convergence is uniform for z in compact subsets ofC \ T. 

Proof. It follows in a straightforward manner proceeding by induction on k and 
using the formula (5.3). ■ 
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A CONSTRUCTIVE BRAUER-WITT THEOREM 
FOR CERTAIN SOLVABLE GROUPS 

ALLEN HERMAN 



Abstract. Division algebras occurring in simple components of group algebras 
of finite groups over algebraic number fields are studied. First, well-known restrictions 
are presented for the structure of a group that arises once no further Clifford Theory re- 
ductions are possible. For groups with these properties, a character-theoretic condition 
is given that forces the p-patt of the division algebra part of this simple component to be 
generated by a predetermined p-quasi-elementary subgroup of the group, for any prime 
integer p. This is effectively a constructive Brauer-Witt Theorem for groups satisfying 
this condition. It is then shown that it is possible to constructively compute the Schur 
index of a simple component of the group algebra of a finite nilpotent-by-abelian group 
tisum the above reduction and 3n Algorithm for computing Schur indices of* simple al~ 
gebras generated by finite metabelian groups. 

Let G be a finite group, k an algebraic number field, and let kG be the group algebra 
of the group G over k. By the Artin-Wedderbum Theorem [CR, 3.28], we know that 
kG is semisimple. A simple component of kG— one of its non-zero two-sided ideals — is 
always a principal ideal of the form kGe, where e is a centrally primitive idempotent of 
kG. Being simple, kGe is isomorphic to a matrix ring over a division algebra D finite- 
dimensional over k. The square root of the dimension of D over its center is known as 
the Schur index of D over k. 

A standard result for computing the Schur index in the above situation in the case of a 
general finite group is the Brauer-Witt Theorem [ W] (see also [Y, page 3 1 ]), stated in Sec- 
tion 2 of this paper. For each prime integer p, this theorem indicates the existence of a p- 
quasi-elementary section of G — see Definition 2 of this paper — that determines the p-part 
of the Schur index related to a given simple component of QG. (Recently, Schmid [Sch] 
has extended this theorem by identifying precise types of p-quasi-elementary groups that 
are the minimal groups one can reduce to using this theorem.) Complete descriptions of 
methods for the computation of Schur indices resulting from p-quasi-elementary groups 
have appeared in [Y], [L], and [Her], the last using an approach based on a paper of 
Janusz [J]. The methods used for these Schur index calculations rely heavily on number 
theoretic information because of the local norm calculations required for computing the 
index. However, the Brauer-Witt Theorem does not lend itself to an algorithmic method 
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of determining sections of G suitable for computing the Schur index, so the methods 
available for computing Schur indices for general groups are not very practical. 

There are several well-known theorems that indicate restrictions on the values of these 
Schur indices [I, Chap. 10]. Occasionally, group structure alone results in restrictions. 
The most notable of these restrictions is Roquette's Theorem that indicates Schur indices 
are at most 2 for nilpotent groups [Roq]. In addition to Roquette's Theorem, constructive 
computations of Schur indices that result from restrictions on the class of the finite group 
have appeared previously. Fontaine has demonstrated that Schur indices in the case of 
finite supersolvable groups can always be constructively computed using the Clifford 
theory reductions described in Section 1 of this paper [F, Prop. 3.4]. Janusz attained a 
constructive reduction for the Schur index question in the case where G has a normal sub- 
group N for which G/N is abelian using methods that involve projective representations 
[J2]. 

The present paper was motivated by the paper of Shirvani [Sh] which attained struc- 
tural invariants for the metabelian case, and the author's work in [Her]. The main results 
of this paper are the following: 

(1) For finite solvable groups G, there is a character-theoretic condition such that for 
any faithful irreducible character of G satisfying this condition, the Schur index of 
the simple component of kG corresponding to this character can be constructively 
computed using pre-determined subgroups of G (Theorem 4). 

(2) The Schur indices of simple components of the group algebras of nilpotent-by- 
abelian groups over algebraic number fields can be constructively computed from 
a knowledge of the multiplication table of the group, using a reduction based on 
the above result (Section 3). 

In order to deal constructively with simple components of kG, we develop the fol- 
lowing somewhat standard notation, based on based on [I] and [Y]. Let Irr(G) denote the 
set of irreducible complex characters of G. For any x G Iit(G), let 

e x = in E xO)xte~')g 

M gee 

denote the centrally primitive idempotent of CG determined by x- It is well known that 
the simple component CGe x is a G-module affording the character x( 1 )x [X Chapter 1 ]. 
For any subfield k of C, let 

«x = E «, 

ae<& 

where & — Gal(k(x)/k). Then a x is a centrally primitive idempotent of kG, and the 
simple component kGa x of kG affords the character x(l) £<7€« X a as a G-module. 

kGa x is isomorphic to a ring of n x n matrices over a division algebra D, for some 
positive integer n. This division algebra D is finite dimensional over its center k(x), the 
field of character values of x over k [I, Exercise 9.15]. Furthermore, the integer m k (x) 
for which 

\D: k( X )| = m k ( X ) 2 
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is the Schur index of \ (or D) over k [CR, page 585]. In what follows, the goal is to find 
a constructive means of computing the Schur index m k {\) that works for as large a class 
of groups as possible. 

1 . Clifford Reductions. Assume now that \ is a faithful character of the finite 
group G. Clifford's Theorem [1,6.2] implies that whenever N <G and the subalgebra 
kNe x of kGe x is not simple, then 



where C is the centralizer in the group G of a centrally primitive idempotent a x of kAfe x , 
and k = \G : C| [P, 6. 1 .7]. The idempotent a\ is the central idempotent of kA/ determined 
by some fixed irreducible constituent A of x„- Furthermore, it follows from [I, 6. 1 1] and 
the fact that C always contains the stabilizer of the character A that x = V> c > f° r some 
irreducible character t/> of C lying over A, and so we even know that e x = a^. Thus 
the division algebra part of kCe x is isomorphic to that of kGe x , and so we might as well 
replace G by C and x by xp. In order to maintain our original assumption that x is faithful, 
we can further replace G by C/(kerxp). 

This reduction of the Schur index problem to one concerning a smaller group is con- 
structive, because the subgroup C = C G (a\) can be obtained (although with possibly 
quite a bit of work) from the multiplication table of G and from some familiarity with 
k. One first computes the character tables of G and N and then uses the formula to write 
down the idempotents e x and all of the e\, A 6 Irr(A0. Once one has found a A for which 
e x e\ is non-zero, one can use the character values of A and the field k to determine a x . 
Then one can determine Cby finding the centralizer in G of a x . Finally, ^ is obtained by 
using the character table of C to find an idempotent such that e x e$ ^ 0, e x e^ ^ 0, 



Only so many non-trivial Clifford reductions of the above type are possible, of course, 
and so eventually one is reduced to the case where kNe x is a simple algebra, for every 
normal subgroup N of G. We will refer to a group that has a faithful character for which 
kGa x has this property as a Clifford reduced group over k. The structure of Clifford re- 
duced groups can be partly inferred from the structure of finite groups with the property 
that every normal abelian subgroup is cyclic, because Clifford reduced groups automati- 
cally have this property. Very precise structural information is available for these groups. 

Proposition . [ Hup, III. 13.10] Suppose that G is a finite group such that every nor- 
mal abelian subgroup of G is cyclic. Then F(G) has a characteristic subgroup F of index 
at most 2. This subgroup F is the central product of a cyclic group U = Z(F) with a 
group E that is the direct product of extraspecial p-groups of exponent p or 4, at most 
one extraspecial p-group for each prime integer p dividing the order ofF. In particular, 
Fj U is elementary abelian. Always, we have F = Cf(G)[Q>(F(G)) ). 



If we further suppose that G is solvable, then further restrictions can be obtained. 



kGe x £ {kCa x e x f* k , 



andxO)=|C7:C|V;(l). 
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LEMMA. Assume that the finite group G is solvable, and that every normal abelian 
subgroup ofGis cyclic. Let F = C^o(o(fl(G))) be the characteristic subgroup of G 
given in the above. Then Z(F) = C G (F). 

Proof. Let U = Z(F). if F = F(G), then since C G (F(G)) C F(G) when G is 
solvable [Sc, 7.4.7], we must have 

U=C G (F)nF=C G (F), 

and we are done. 

If F has index 2 in F(G), then note that the Fitting subgroup of C G (F) must be FD 
C G (F) = U, because the Fitting subgroup of C G (F) is a normal nilpotent subgroup of G 
and hence lies inside F. 

Suppose C G {F) properly contains U. Let X/ U be a chief section of G with X C C G (F). 
Because G is solvable, X/U 'isa non-trivial p-group, for some prime p. Since U is the 
Fitting subgroup of C G (F), U is also the Fitting subgroup of X, and so A' cannot centralize 
U. This contradicts the assumption that X centralizes F, and so we conclude that U = 
C G {F). m 

The above lemma allows one to come to certain conclusions about the structure of 
certain sections ot tne group u. 

PROPOSITION. Let G be a solvable group, and suppose that every normal abelian 
subgroup ofGis cyclic. LetF = C^o(o(F(G))). Then if we let U = Z(F) and C = 

C G (U), we have that G/C is abelian, and C/F is isomorphic to a solvable subgroup of 
the direct product of finite symplectic groups of the form Sp(2n,p), for any prime integer 
p and power n such that \F : U\ is divisible by p 2 ". 



2. A Condition on Characters. In this section, we assume that G is a finite solvable 
group that is Clifford reduced over an algebraic number field k with respect to a faithful 
irreducible character x- Our goal is to find conditions which guarantee that a further 
constructive reduction of the Schur index problem is possible. 

Fix a prime integer/? throughout. The following definition will make it easier to state 
our results. 

Definition 1 . Let x be a faithful irreducible character of a finite group G, and let k 
be an algebraic number field for which k = k(x)- Let e = exp(G). The p' -splitting field 
ofYGe x over k is the unique subextension K of k(Q) containing k such that 

[k(Cr) : K] is a power of p, and 
[K : k] is relatively prime to p. 

Because cyclotomic extensions are always abelian Galois extensions when the base field 
has characteristic zero, it follows that the K defined above is the field fixed by the unique 
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Sylow p-subgroup of Gal(k(C)/k), and so K is unique. The property of this K that we 
need is that the Schur index of the division algebra part of 

KGe x = K<S> k kGe x 

will be precisely the p-part of /w k (x)- 

In order to state the Brauer-Witt Theorem, we need the following definition. 

Definition 2. A finite group is called p-quasi-elementary if it is isomorphic to the 
split extension of a cyclic group of order relatively prime to p by a finite p-group. 

THEOREM (BRAUER-WITT). Let \ be an irreducible character of a finite group G. 
Let k be an algebraic number field satisfying k(x) = k. Let K be the p' -splitting field for 
kGe x over k 

Then there exists a p-quasi-elementary section H of G and a character £ e \n(H) 
such that 

m K (0 = ("k(x)) p . 



Proofs of the Brauer-Witt Theorem are based on Brauer's Induction Theorem [CR, 
1 5.9], which is used to establish the existence of the section H of the correct type and the 
desired character £ [Y, Chapter 3]. This approach does not lead by itself to an algorithmic 
method of finding such a section H from a knowledge of the subgroup structure of G, and 
thus has limited practical applications. In what follows, we will show that the Brauer- 
Witt Theorem can be made constructive when x satisfies a certain character-theoretic 
condition. 

Assuming that the solvable group G is Clifford reduced over k with respect to a faith- 
ful irreducible character x, let K be the p'-splitting field for kGe x over k. Since the p-part 
of the Schur index of kGe x is exactly the Schur index of KGe x , we may replace k by K. 

The first step is to re-do the Clifford Theory reductions of the previous chapter with 
respect to the new field K. Assume we have completed this process, so that for every 
normal subgroup N of G, KNe x is a simple algebra. In particular, every normal abelian 
subgroup of G is cyclic, so G has normal subgroups F, U> and C satisfying the following 
conditions of the previous section: 

(1) F is a characteristic nilpotent subgroup of G having index at most 2 in F(G); 

(2) U = Z(F) is cyclic, and F/ U is elementary abelian; 

(3) F is the central product of a group E with U, where £ is a direct product of ex- 
traspecial ^-groups having exponent q (or 4 when q is 2), for some prime integers 
q\ and 

(4) C = C G (U). 

We now establish various character identities for irreducible characters of the above 
subgroups of G. 
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PROPOSITION 3. Let K be the p' -splitting field for the simple algebra kGe x . Sup- 
pose that KGe x has been fully reduced using Clifford theory, and let F, U, and C be the 
subgroups of G defined above. Let She a transversal ofC in G. Then 

(i) x v = k Ejes A 5 , for some faithful irreducible character XofU and some integer 
k>0; 

(ii) G/C^Ga\(K(X)/K); 

(Hi) x ~ ip G , for some xj) € Irr(C); 

0 V ) X F = d ^-ses V s >f or some faithful irreducible character ip ofF, and some integer 
d>0. 

(v) <p and ip may be chosen so that <p v = fX, with/ 1 = [F : U], and i/v = dtp; 

(vi) K(X) = K{<p) = Kty) = K(Q), where u — \V\. 

PROOF. Since U < G, the algebra KUe x is simple under our assumptions. Thus 

where A is some irreducible character of U, Q = Gal(AT(A)/K), and k is some positive 
integer. Because all of the characters X" lying under \ are Galois conjugate, all of their 
kernels are the same. Thus 

kerA = f| kerA ff = kerx,, = (/nkerx = UHi = 1, 

so A is a faithful irreducible character of U. Since A is a faithful linear character, the 
stabilizer of A in G must exactly be the centralizer in G of the cyclic group V; namely, 
C. The character-theoretic version of Clifford's Theorem gives the decomposition 

where 5 is a transversal of C in G. This proves (i). (iii) also follows because \ is always 
induced from the stabilizer in this situation [I, 6.1 1]. Mapping G onto permutations on 
the set {X° : a £ Q} gives a natural isomorphism of G/C with Q, namely gC a(g), 
where o(g) is defined by A* = A^. This proves (ii). 

Now, x = V> c , for some t/> G Irr(C) such that 0 (; = kX. This forces K(X) C K($). 
If K(X) ^ K(ip), then there is a galois automorphism fixing A but not xp, and so there is 
more than one rp lying over A and inducing x- This would contradict [I, 6.1 1(c)]. (This 
method originates in [C, Lemma 1.1].) Thus K(X) = K(xp). 

As in the case of(J,F<G implies that 

X, =dJ2 <p t , 

where y is some irreducible character of F, 'H = Gal(A:(^)/A:), and d is some positive 
integer. As above, we can show that <p is faithful. Because F is nilpotent of class 2, it 
follows from [I, 2.31] that any faithful character of F vanishes off Z(F) = U. Thus we 
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can choose </? G Irr(F) so that ip u = /A, with/ 2 = [F : U]. Since </? must vanish off U, 
we have that K(<p) = K(\), and the stabilizer of <p is exactly C. This proves (vi), and (iv) 
follows as in the above with A replaced by <p. 

As tj) F = d<p s , for some s G S, the fact that {xl) F ) u is a multiple of A implies that 
ip F = d<p. Statement (v) follows. ■ 

One final remark is needed before the proof of the main result of this section. From 
our assumption concerning K, we see that by part (ii) of the above proposition, G/C must 
be a p-group. Thus for any Sylow p-subgroup of G, we have G = CP. Since non-identity 
elements of a transversal of C in G must lie outside C, we can choose the transversal S 
such that S C P, for any previously chosen Sylow /^-subgroup P. 

THEOREM 4. Let K be the p' -splitting field for the simple algebra kGe x . Suppose that 
KGe x has been fully reduced using Clifford theory, and let F, U, and C be the subgroups 
of G defined above. Let Pbea Sylow p-subgroup ofG for the fixed prime p. Suppose that 

H= UP=Up,xiP t 

where is the p-complement of the cyclic group U. 
Then 

(i) H is a p-quasi-elementary subgroup of G; 

(ii) Ifxp F = dtp, with (d,p) = 1, then there exists a £ 6 Irr(fl) such that (x H ,Q ^ 
Omodp. 

(Hi) Ifip F = tp, then there exists a£e Irr(//) such that (x„, 0 ^ 0 modp andK(£) = 
K. 

PROOF, (i) is clear, since U is a cyclic normal subgroup of G. 

To prove (ii), we let Q be the normal subgroup HHF of G, and we compare expressions 
for the characters (\ F ) Q and (x H ) Q - Suppose that we have chosen a transversal S of C in 
G so that SC. P. Then we have that 

S£S 

= £ V ft' 
y/W^Aks ' 

for some 9 e Irr(Q) that is fully ramified with respect to U and lies over A. On the other 
hand, suppose 

r 

for some positive integers c\ , . . . , c r , and irreducible characters £i , . . . , £ r of Q. Since the 
Galois conjugates of 6 are the only irreducible characters of Q lying under \, they are 
also the only irreducible characters of Q that lie under any of the characters . . 
Since {9 s : s € 5} is exactly the set of Galois conjugates of 9, and each s € H, we 
conclude that for each / = 1 , . . . , r, we have 
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for some integer > 0. Therefore 

r 

(x w ) e = £ c /(S<V 

i=l 5(;S 

i=l s€S 

Comparing the two expressions for \ Q - (\ F ) Q = (x„) 0 > we conclude that 

By our assumption that d is relatively prime to the second number has to be relatively 
prime to p. This implies that at least one of the numbers c,/** 1 in the sum on the left has to 
be relatively prime to p. Without loss of generality, assume ci/?* 1 is relatively prime to p. 
Then we must have k\ = 0 and c\ = (£,, \ H ) ^ 0 mod/7. This £ = £\ is the irreducible 
character of H required for (ii). (Note that we have also shown that for this character £, 

ses 

To prove (Hi), note that the assumption that \p F = tp is equivalent to 

KFa^^KCa^, 

since K{$) = K(tp) and both algebras have dimension ^(l) 2 = y?0) 2 over their centers. 
Our assumption on G then gives KFe x = KCe x , and so 

KCe x HKHe x = KFe x C\KHe x . 

The subalgebra KHe x need not be simple because H need not be normal in G. However, 
we know from \ H = £» c /£< that 

KHe x = ®iKHa^e x , 

because KHe x is a homomorphic image of KH as an algebra. (The simple components of 
KH that occur in this sum will be exactly those whose associated irreducible characters 
are constituents of x H ) Let 8 € Irr(@) be as above. Since 8 vanishes off U, it follows 
that I H (0) = H(~) C. Let R = HC\ C, and note that FR (F is a Sylow p-subgroup of C/F. 

Now let £ € \rr(H) be the character found in the proof of part (ii). From the remark 
that 

** = E^ 

it follows from R = I H (8) that there is a character i] € hr(R) such that £ = rj H and 
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Consider the subalgebra KRc\. We have that KRe x C KCe x , so there are algebra 
inclusions 

KQe x C KRe x C KFe x . 

Since y> e =f0 with / = : Q\, we know that 

[KFe x : KQe x ] = \F : Q\ 

is relatively prime to p. Thus the index of KQe x in the subalgebra A7?e x of KFe x has to 
be relatively prime to p. But R/Q is a p-group, and K is the p'-splitting field, so [KRe x : 
KgeJ must be a power of p. This forces KRe x = K£?e x . 

Note that each of the characters occurring in the above sum is faithful on Q by the 
formula = p* 1 Y.s& ■ Since K£te x is a simple subalgebra of KHe x , this means that 
each of the maps KQe x — ► KQa^e x has to be an isomorphism. (Hence we can interpret 
the inclusion 

KQe x -» ®,A7/a^ e x 

as a diagonal embedding.) Since A7?e x = K(2e x , we have that KRa^e x = KQa^e x is 
isomorphic to the simple component KRa,, of The center of KRa n is an isomorphic 
copy of K(i)\ and so since KQe x ^ KQa^e x ^ KRa n , we conclude that K(B) = K(ij). 

Finally, note that since £ is induced from I H (B) = R, £ vanishes off R. Since H/R acts 
as galois automorphisms on the subfield K(0) = KUa^e x , H / R acts as galois automor- 
phisms on K(i)) = Z(KRa,,e x ). Thus K(0 ^ Z{KH ai e x ) is the subfield of K(9) fixed by 
H /R. Of course, this is exactly as required. ■ 

Corollary 4.1. Under the conditions of Theorem 4, ij we have if) F = if, then 
K(X))„ = "k(0. 

Proof. The conclusion of Theorem 4, part (iii) is exactly what is needed to apply 
[Y, Corollary 3.8], from which we get m K (x) = The corollary follows because 

Schur indices of characters of p-quasi-elementary groups have to be powers of p and 
[K : k] is prime to p, which forces m K (0 = m k (0 by [Rit, Theorem 2(c)]. ■ 

We are now motivated to find conditions for the group G that ensure that the character 
<p of F extends to its stabilizer C once G has been reduced using Clifford theory. Exact 
conditions for this are unknown to this author, although a considerable amount of litera- 
ture is available on conditions for character extension. (See, for instance, [G] and [12]). 
Of course, if C/Fis cyclic, then it is well known that <p extends to C [I, 1 1.22]. 

3. The Nilpotent-by-Abelian Group Case. In this section, we assume that G is 
a finite nilpotent-by-abelian group that is Clifford reduced with respect to its faithful 
irreducible character We conclude the paper by showing that for such a nilpotent-by- 
abelian group, the same p-quasi-elemcntary group H as in Theorem 4 above can be used 
to compute the Schur index of \. 

From the structure results in Section 1, we first note that the following holds for 
nilpotent-by-abelian groups. 
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PROPOSITION 5. Suppose that G is a finite nilpotent-by-abelian group and \ G 
Irr(G). Then kGe x Clifford reduces to a simple component of the group algebra kG, 
where G is a nilpotent-by-abelian group of derived length at most 3. 

PROOF. Assume that kGe x has been fully reduced using Clifford theory, and let the 
subgroups F, U, and C be as above. Since the class of nilpotent-by-abelian groups is 
closed under taking sections, we still have that G is nilpotent-by-abelian. We know that 
F = F(G) n C and G/C is abelian, so our assumption that G/F(G) is abelian implies 
that 

G' CF(G)nC = F, 

and so G/F is abelian. The proposition follows because F is a nilpotent group of class 
at most 2. ■ 

(We remark that if G is any solvable group, the Clifford theory reductions on kGa x 
end in a simple component of the group algebra of a group having derived length at most 
the derived length of G/F(G) plus 3.) 

Now, let K be the //-splitting field for kGe x , and suppose that KGe x has been fully 
reduced using Clifford Theory. Let F, U, and C be the characteristic subgroups of G 
determined in Section I, and let tp G Irr(F), A € ln(U), and i> e Irr(C) satisfy the 
conclusions of Proposition 3. By the proof of Proposition 5 we know that G/F is abelian. 

LEMMA 6. Under the assumptions of the preceding paragraph, there exists a normal 
subgroup B and an irreducible character 3ofB such that 
(0 F C B C C; 
(ii) X = P°. 1> = ffl and 
(Hi) 0, = <p. 

PROOF. Because G/F is abelian, G is a relative M-group with respect to F [I, 6.22], 
which means that for every character x' G Irr(G), there is a subgroup B' containing F 
and a /3 G Irr(fl') such that /? /C = x' and 3' F G Irr(F). Assuming B and 8 satisfy this for 
the character x, we see that x = 3 G . Since G/F is abelian, B has to be a normal subgroup 
of G. The assumption that KGe x is fully reduced using Clifford theory then implies that 
KBe x is simple, hence KBe x * KBa 0 . Since x = P 0 , B is the stabilizer of 3 in G, and so 

G/B & Ga\(K(3)/K). 

Since the G-conjugates of A are the only irreducible characters of F lying under x, we 
must have 3 F = <p*, for some g G G. This ip g is thus invariant in B. Since C is the 
stabilizer of every ip* in G, this implies that B C C. As 8* ' also induces x, replacing 3 
by 3 s ' 1 gives us the condition 3 F = tp. Since 8 C is irreducible and lies over \p t we get 
that 8 C = V> because 0 G Irr(C) is unique with this property. All i 
now been shown. 

The next step is to reduce to a group having a normal p-complement. 
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LEMMA 7. Suppose that G is a nilpotent-by-abelian group with faithful irreducible 
character \. Assume KGe x has been fully reduced using Clifford theory, where K is the 
p' -splitting field of Q(x)Ge x . Let M be a subgroup of G so that M/F is the Sylow p- 
subgroup of G / F, F as above. Then KMe x = KGe x . 

Proof. Let B be as in Lemma 6. Since G/B Gal(AT(^)/A:) is a p-group, we 
can choose a transversal X of B in G consisting of elements of M. Since <p extends to 
3 € Irr(ff), we see that 

KBe x * KFe x 0 K(lfi) K(0), 

because 8( 1 ) = <p( 1 ). Let D be the subgroup of B for which Dj F is a Sylow p-subgroup 
of B/F. Note that KDe x must be simple because D < G. Since ip extends to D, the same 
reasoning as above shows that 

KDe x * KFe x K(0 D ). 

Now we see that [KBe x : KDe x ] = [K(J5) : K(J3 D )], which is a power of p by our 
assumption on K. However, since [KBe x : KDe x ] divides \B : D|, it is relatively prime 
to p. We conclude that KDe x = KBe x . Since 0 G = x, we have 

KGe x = M^)x 
= ® xeX (KDe x )x 
= KMe x , 

since X is also a transversal of D in M. ■ 
Using Lemma 7, we now assume G/F is a //-group by replacing G by M if necessary. 
If P is a Sylow ^-subgroup of G, then we have that G = FP, and so since the p'-part of 
F is normal in G, G can be written as 

C = JVxP, 

with the order of W being relatively prime to P. Together with the assumption that KGe x 
is fully reduced using Clifford theory, this presentation forces the ^-part of the subgroup 
Fto be cyclic. (This observation follows from [MW, Corollary 1.10 (iii)] and the fact 
that every chief section of such a group G that lies over N must have prime order p.) 
Suppose that B is the normal subgroup of G identified in Lemma 6 that possesses an 
irreducible character [i extending (f and inducing \ . The next step is to arrange that the 
field of character values K{j5) is a cyclotomic extension of the p'- splitting field K. 

Let e = exp(B). Since B/F is a p-group, we must have e = p c exp(F), for a non- 
negative integer c. From our knowledge of the structure of F we find that exp(F) is either 
u = \U\ or 2m. Furthermore, exp(F) can be 2u only when p is odd, 4 divides \N\, and 4 
does not divide \ U\. The inclusions 

K(<f) = C K(J3) C K(Q 
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are obvious, with both [£(£) : K(Ji)] and [K(fi) : K(C)] equaling powers of the prime 
p. Since e is either p c u or 2p c u, we have that whenever p is an odd prime, the subfields 
of K((^c u ) (or K{Cpr u ,<^) when e = 2p c u) that have index a power of p over K((^) are all 
of the form K(<^/„)> f° r some integer i such that 0 < I < c. (This is immediate because 
the p-part of the respective Galois groups are cyclic.) So in the case where p is odd, 
K(0) = K^,,) for some integer i, and i can be determined easily from e and \G : B\. 

If p = 2, then the subfields of K((jc u ) lying over K(Q) do not have to be of the form 
^(C2'u)> for 1 < £ < c, unless 4 divides u. If 4 divides u, then we determine that K(J3) 
is a cyclotomic extension of K exactly as above. Assuming that 4 does not divide u, 
we "inflate" the group G to a new group, G, and arrange that 4 divides the order of the 
necessary cyclic normal subgroup of G. If | U\ is even, define G to be the central product 
of G with the abstract group 

D 8 := (x,y\x 4 =\=y 2 ,xy = x->) 

— a copy of the dihedral group of order 8. If \U\ is odd, then define G to be the direct 
product of G with the same group D 8 . KG has a simple component that is isomorphic to 
the tensor product 

KGe x ® K K 2x2 . 

This simple component is associated with a faithful representation of G. Not every nor- 
mal abelian subgroup of G is cyclic, since D 8 does not have this property. However, if we 
set F = (F,x), U = (U,x), and B = (B,x), then any faithful character associated with 
the above simple component is induced from a faithful irreducible character /3 of B. Since 
K0) is a subfield of K(& u ) containing AXCCO, K0) has to be a cyclotomic extension 
of K, as required. Since the Schur index of the new simple component is the same as 
that of KGe x , it suffices to do our computations based on the slightly larger nilpotent-by- 
abelian group G, with the above definitions for the corresponding subgroups and their 
characters. 

Once we have that the character field K(J3) is a cyclotomic field, we can "inflate" the 
group G in order to apply Theorem 4 directly. (In the special case, replace G by G, B by 
B, (3 by 0, and etc.) Let 2 be a root of unity in the center of KBe x so that 

Z(KBe x ) = K(z). 

Let G be the finite subgroup of the group of units of KGe x generated by z and G. Then it 
is easy to see that KGe x = KGe x . Furthermore, the subgroups F = (z,F), 0 = (z), and 
B = (z,B) all generate simple subalgebras of K&e x , as do all normal subgroups of (5 
lying between 0 and F. Because we have arranged that K(fi) = Z(KFe x ), we now have 
KFe x = KBe x . This allows us to use the proof of Theorem 4 (without the assumption that 
every normal abelian subgroup of G is cyclic in the special case), and conclude that there 
is an irreducible character of H = UP that has the same Schur index as that of KGe x , 
for some Sylow p-subgroup P of G. Thus we have constructively reduced the problem 
of determining the Schur index of KGe x to determining the Schur index of a simple 
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component of the group algebra of a p-quasi-elementary group. Calculations necessary 
for computing the index of such an algebra have been described in [Y], [L], and [Her]. 

This completes the outline of the algorithm for computing Schur indices of simple 
components of a group algebra of a nilpotent-by-abelian group over an algebraic number 
field. We conclude this section with the following example that indicates the limitations 
of this process for general solvable groups. 

For example, let E be an extraspecial group of order 27 and exponent 3. The outer 
automorphisms of E that fix the center of E form a group isomorphic to SL(2, 3). Let G be 
the semi-direct product E >J SL(2, 3) in which SL(2, 3) acts as these outer automorphisms. 
The irreducible characters of G which lie over a fixed faithful irreducible character ip of 
E are in one-to-one correspondence with characters ofSL(2, 3). These characters are all 
faithful. Simple components of Q((j)G corresponding to these characters do not reduce 
using Clifford theory, in fact, E = Fand G = C in the above notation. There are three 
characters of G that extend tp, corresponding to the linear characters of SL(2, 3). For 
these characters \h « = 1,2,3, a reduction using Theorem 4 is possible, and it shows 
that the 2- and 3-parts of /wq«j)(X/) are Schur indices of simple components of rational 
group algebras over the groups C3 x Q» and E x C3, respectively. Since the quaternion 
component of C3 >i Q% is not the one in question, and E x C3 is a 3 -group, we must have 
ntoixi) = 1 for i — 1,2, 3. On the other hand, for the three characters of G lying over 
<p that satisfy (X4) F = 2</?, (xs) F = 2<p, and (X6) f = 3</>, the reduction fails because 
the hypothesis is not satisfied. Of course the group H can be computed, but we cannot 
guarantee the existence of a £ e ln(H) that will have the right Schur index. 
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INDUCTION AND RESTRICTION OF tt-PARTIAL 
CHARACTERS AND THEIR LIFTS 

I. M. ISAACS 



Abstract Let G be a finite 7r-separable group, where n is a set of primes. The 
w-partial characters of G are the restrictions of the ordinary characters to the set of ir- 
elements of G. Such an object is said to be irreducible if it is not the sum of two nonzero 
partial characters and the set of irreducible 7r-partial characters of G is denoted MG). 
(Ifp is a prime and n = p', then l w (G) is exactly the set of irreducible Brauer characters 
at p.) 

From their definition, it is obvious that each partial character <p G I„(G) can be 
"lifted" to an ordinary character \ e Irr(G). (This means that y is the restriction of x to 
the 7r-elements of G.) In fact, there is a known set of canonical lifts B„(G) C Irr((7) for 
the irreducible 7r-partial characters. In this paper, it is proved that if 2 ^ w, then there is 
an alternative set of canonical lifts (denoted D„(G)) that behaves better with respect to 
character induction. 

An application of this theory to M-groups is presented. If G is an M-group and S C 
G is a subnormal subgroup, consider a primitive character 9 € Irr(5). It was known 
previously that if |G : S| is odd, then B must be linear. It is proved here without restriction 
on the index of S that 0( 1 ) is a power of 2. 

1 . Introduction. This paper is intended to serve a double purpose. We obtain a few 
new results in the 7r-character theory of 7r-separable groups and we use some of these to 
study the character theory of subnormal subgroups of M-groups. 

Consider M-groups first. Suppose that 5<mC, where G is an M-group, and suppose 
7 e Irr(5) is primitive. In [3], we showed that if S has odd index, then 7 must be linear 
and it is a consequence of our work here that 7 must also be linear in the case where S 
has odd order. In fact, we have more. 

THEOREM A. Let S « G. where G is an M-group. If 7 € lrr(5) is primitive, then 
7(1) is a power of 2. 

We mention that the case of Theorem A where S < G is the main theorem of [1], but 
the result for subnormal subgroups does not follow from this. Our actual result here is 
an even stronger necessary condition for a group S to be subnormally embedded in an 
M-group, but unfortunately, our condition does not imply that the S must be an M-group, 
even when it is known to have odd order. 

We turn now to 7r-character theory, with a brief review of definitions and basic re- 
sults. (The reader may wish to consult the expository papers [8], [5] and [7] for more 
information and for proofs of some of the key theorems.) 
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Fix a set n of prime numbers and let G be a 7r-separable group. The it-partial char- 
acters of G are the restrictions of the ordinary characters to the set G° of 7r-elements of 
G. (We write \° to denote the restriction of x € Char(G) to G°.) A 7r-partial character 
is irreducible if it cannot be written as a sum of two nonzero 7r-partial characters and we 
write h(G) to denote the set of all irreducible 7r-partial characters of G. If x £ Char(G), 
it is immediate that we can write x° = Y.ay<p, where ip runs over l n (G) and the coef- 
ficients a^ are nonnegative integers. (In fact, these coefficients are uniquely determined 
by x because l n (G) is actually a basis for the complex vector space of class functions 
defined on G°.) Given <p G \*{G), we can certainly write <p = x° for some character x of 
G, and in this situation it is clear that x must lie in Irr(G). In other words, each irreducible 
vr-partial character of G can be lifted to an ordinary irreducible character. 

It is possible to 'predetermine' a complete set of lifts for the irreducible 7r-partial 
characters of a 7r-separable group G. There is, in other words, a canonically defined subset 
B^G) C Irr(G) such that the map x X° is a bijection from B„(G) onto I T (G). (The 
definition and principal properties of the set B n (G) appear in [2] and an exposition of this 
material can be found in [7].) 

The function B„( ), which assigns to each 7r-separable group G a particular set of 
irreducible characters of G, behaves well with respect to normal subgroups: if x G B*(G) 
and N <i G, then all irreducible constituents of the restriction x/v lie in B„(A/). (As is 
explained in [8], this fact about the set-valued function B ff ( ) is crucial in establishing the 
analog of Clifford theory for the irreducible 7r-partial characters.) 

There are some respects, however, in which the behavior of the function B*( ) is 
not under good control. Suppose, for example, that H is an arbitrary subgroup of G. If 
X G B n (G), then even if we know that x« is irreducible, we cannot always conclude 
that x// lies in B*(//). Similarly, if we start with xp G B n (H), where ip G is known to be 
irreducible, it does not follow that xP G lies in B„(G). 

In the case where G has odd order, it is an easy consequence of Lemma 3. 1 of [6] that 
the restriction and induction conclusions of the previous paragraph actually are valid. In 
fact, an even stronger induction result holds: given that \G\ is odd and that ip G = \ € 
Irr(G), then V> € B X (H) if and only if x G B„(G). (The additional information here, of 
course, is the 'if assertion, which appears as Theorem 8.6 of [8].) 

For groups of even order, one can use Lemma 3.3 of [6] to prove weak forms of the 
restriction and induction theorems for B„( ), but only when 2 G ir. In the case where 
2 £ 7r, on the other hand, strong replacements for these theorems are available, provided 
that we are willing to make certain changes. Specifically, we must replace B n ( ) by a 
suitable alternative function D„( ) that also assigns a certain subset of Irr(G) to each n- 
separable group G and we must work with a twisted form of character induction called 
7r-induction. 

Definitions and properties of 7r-induction and of the function D w ( ) can be found in [4], 
and we review some of this material in Section 2, following this introduction. We prove 
here that D K (G) (like B n (G)) is a set of lifts for the irreducible 7r-partial characters of G. 
In general, D^G) can be different from B„(G) and we stress that D^G), unlike B„(G), 
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is defined only when 2 £ tt. (We mention that the definition of D n (G) was suggested by 
unpublished work of E. C. Dade.) 

THEOREM B. Let G be n-separable, where 2 $ ir. Then the map \ •— » X° <feymes a 
bijection from D T (<7) on/o U(G). 

Using Theorem B, it is easy to deduce the following. 

COROLLARY C. IfG has odd order and 2 g n, then D„(G) = B„(<7). 

By Theorem 7.10 of [4], the function D^( ) (like B n ( )) respects restriction to normal 
subgroups, but unlike B^( ), it is also reasonably well-behaved upon restriction to non- 
normal subgroups. 

Theorem D. Let H C G, where G is n-separable and 2 g n. If \ G D„(G) and 
"4> = XHis irreducible, then t/> G D„(H). 

In view of Corollary C, we see that for odd-order groups, Theorem D implies the 
restriction theorem for B 7r ( ), to which we referred earlier. 

The induction theorem for the function D„( ) is known. It was first proved in unpub- 
lished work of Dade and it appears as Theorem 7.3 of [4]. For comparison with Theo- 
rem D and because together with Theorem B, we use it to prove our M-group theorem, 
we state this result below as Theorem E. 

We must first give at least a cursory description of 7r-induction, however. (More detail 
can be found in Section 2, and still more appears in [4].) If 2 & n and G is 7r-separable, 
then 7r-induction from a subgroup H C G is a certain map 0 ■— * B* G from Char(//) to 
Char(t7). Like the ordinary induced character 0 G , the 7r-induced character &* G has degree 
equal to \G : H\ 0(1). More generally, 0* G (jc) = 0°(x) for all odd-order elements x e G. 
(For odd-order groups, therefore, 7r-induction is the same as ordinary induction, and so 
by Corollary C, the induction theorem for B*( ) in odd-order groups (to which we referred 
earlier) is a consequence of Theorem E.) 

THEOREM E. Let H C G, where G is n-separable and 2 & ic. Suppose xp G Irr(//) 
and that X = V G is irreducible. Then \ E D„(G) iffif> € D„(H). 

We mention that the 'if part of Theorem E is immediate from the definition of the 
function D„( ) and the properties of 7r-induction; it is only the 'only if part that requires 
work. 

Since we are assuming that 2 g n and we know that 9° and Q* G agree on elements 
of odd order, we sec that these characters agree on the set G° of 7r-elements of G. It 
follows that (0* G )° = ((Pf for all characters 9 of //, and so from the point of view of 
7r-partial characters, 7r-induction is exactly the same as ordinary induction. It is induction 
of 7r-partial characters that we consider next. 

Recall from [8] that a character \ € Irr(G) is supermonomial if whenever x = V ,G > 
where V is a character of a subgroup of G, then xp is monomial. Analogously, a partial 
character tp G I W (C7) is monomial if it is induced from a linear 7r-partial character of a 
subgroup and xp is supermonomial if every Tr-partial character that induces it is monomial. 
Our principal supermonomiality result, used to prove Theorem A, is the following. 
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THEOREM F. Let G be p-solvable, where p is an odd prime. Choose ip £ l p (G) and 
let x be the unique lift of <p in D P (G). The following are then equivalent. 

(i) if is monomial. 

(ii) x I s monomial. 

(Hi) Both (f and \ are super monomial. 

The fact that monomial characters in D P (G) must be supermonomial is a generaliza- 
tion of Theorem 10.2 of [8], which is the odd-order case of this result. (In [8], the theorem 
is stated for B P (G), but we know that B P (G) = D P (G) when |G| is odd.) 

Theorem A is not the only result about primitive characters of subgroups of M-groups 
that can be proved using techniques of this paper. Instead of discussing other theorems 
of this type here, we refer the reader to [9]. 

2. Sign characters and twisted induction. In this section we recall some def- 
initions and facts from [4], and in particular, we review the definition of the subset 
V*(G) Q Irr(G). The key to this is the -^-standard sign character 6 {G:H) , defined for 
each subgroup H C G, where G is 7r-separable and 2 £ tt. This is a linear character 
of H that has values ±1; it is determined by the action of H on the 7r-factors of an H- 
composition series for G. (We refer the reader to Section 2 of [4] for the full definition of 
6(G:H)-) Among the properties of the 7r-standard sign character are the following, which 
are sufficient to determine it uniquely. 

LEMMA 2. 1 . Given H C G, where G is ^-separable and 2 g it, we have the follow- 
ing. 

(a) If\G : H\ is a tt 1 -number, then 8 {G:H) = \ H . 

(b) If\G : H\ is a ir-number and H is a maximal subgroup, then 6(g-.h) is the permu- 
tation sign character of the action of H on the right cosets of H in G. 

(c) IfHCKCG, then bajjf) = (S(G:K))h6(K-.H)- 

PROOF. Parts (a) and (c) are respectively Theorem 2.5(c) and Theorem 2.5(b) 
of [4], while (b) is immediate from Corollary 2.9(a) of that paper and the fact that 
the sign character of an action is exactly the determinant of the associated permutation 
representation. ■ 

Continuing to assume that G is 7r-separable, with 2 # it, we recall from [4] that if 9 G 
Char(//), then the -^-induced character Q* G of G is the character (89) G , where 5 = 6{gj{)- 
Since 6(G:H) » s trivial if |G : H\ is a V-number, we see that 7r-induction from subgroups of 
Tr'-index agrees with ordinary induction. Also, for arbitrary subgroups H C G, if x € G 
has odd order, then 0 w( '(x) = ^(jc). (This is because b(y) = 1 for every element y of H 
that is conjugate in G to x.) 

In general, 7r-induction enjoys many of the familiar properties of ordinary induction. 
It is transitive, for example. In other words, if//CKCGand0€ Charf//), then 
QpKyG _ q*g ( This is an i mme( iiate consequence of Lemma 2.1 (c).) Somewhat more 
interesting is the situation where G = XY for subgroups X and Y. If 9 G Char(K), it is 
a well-known consequence of Mackey's theorem that (0°)* = {9 X nvY- The analog of 
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this formula for 7r-induction would be immediate if we knew that (jb(G:Y))xnY = ^x.xr\Y), 
but in [4], this was proved only under the additional assumption that one of X or Y is the 
product of X D Y with a normal subgroup of G. In fact, it is not very difficult to prove 
the general case. (In order to give a self-contained proof here, we do not assume the part 
of this result that appears in [4].) We require the following lemma, which, except for its 
dependence on the odd-order theorem, is entirely routine. 

LEMMA 2.2. Let G be ir-separable, where 2 $ it, and suppose (V((7) = 1. Then all 
core-free maximal subgroups ofG are conjugate. 

Proof. Let N be a minimal normal subgroup of G. Then A 7 is a 7r-group, and thus is 
of odd order and hence is an abelian p-group for some prime p G n. If X is a core-free 
maximal subgroup of G, then N X and thus NX = G and N n X = 1 . It follows that 
C G (N)nX<i G, and we conclude that C G (N) = N. 

lfN=G then X = 1 and there is nothing to prove. We can thus assume that N < G 
and we let K/ N be a chief factor of G. We claim that K/N is a p'-group. This is certainly 
true if K/N is a 7r'-group, and so we can assume it is a 7r-group. It thus has odd order, and 
hence it is an abelian ^-group for some prime q. But q ^ p; otherwise, AT is a /7-group 
and N C Z(/Q, which contradicts the fact that N is self-centralizing. 

We see now that XHK is a p-complement in K and that* = N G (XnK). Since all 
/^-complements in K are conjugate, their normalizers in G are also conjugate, and the 
result follows. ■ 

LEMMA 2.3. LetX, YCG, where G is -^-separable and 2 £ tt. Assume thatXY = G 
and write D = XCiY. Then 

(a) 0(G.Y))d = S (X:D) . 

(b) (6(GJf))0 = 5(Y.D). 

(c) If 9 G Chaitn then {Q" c ) x = {0 D y x . 

Proof. We observe first that (a) and (b) imply each other for each choice of sub- 
groups X and Y with XY = G. To see this, we argue that since 5 {Y: d) is nonvanishing, 
the assertion of (a) is equivalent to the equation 6 IX : D Ay:D) = ($(G:Y))d6(Y.d)- Since the 
right side of this equation is equal to 8 (G:D) by Lemma 2. 1(a), it follows that (a) is true iff 
t>(G.D) = 8(X:D)f>{Y.Dh which is symmetric in A' and Y. It follows that (b) is also equivalent 
to this equation, and hence (a) and (b) are equivalent to each other. 

We now prove parts (a) and (b) by induction on the index \G : D\. If X = G, then 
D = Y and (a) is trivially true, and it follows that (b) holds too. Similarly, (a) and (b) 
hold if Y = G, and thus we may thus assume that X and Y are each proper in G. 

Suppose that X < H < G for some subgroup H. Observe that HY — G and that 
XE = H, where we have written E = HHY. Since H > X, we see that E > D and thus 
\G : E\ < \G : D\. Also \H:D\<\G:D\ since H < G, and hence two applications of 
the inductive hypothesis yield that% :0) = (S^.e^d = ((&(G.Y))e) D - Thus (a) holds, and 
so (b) holds too, and similarly, both (a) and (b) hold if Y is not maximal in G. 

We can now assume that each of X and Y is a maximal subgroup of G. In particular, 
since G is 7r-separable, each of the indices \G : X\ and \G : Y\ is either a 7r-number or 
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a 7r'-nuiTiber. Suppose that one of these indices (say |G : Y\) is a V-number. Then also 
\X : D\ is a ^-number and hence 8(g.y) = 1 k and b(x.D) — Id and assertion (a) follows. 
Since (a) and (b) both hold if either of the two indices |G : X\ or |G : Y\ is a /-number, 
we can assume that both indices are 7r-numbers and hence \G : D\ is a 7r-number. 

Now let M = corec(D) and suppose that M < corec(10- Writing N = corec(10» we 
see that N X and thus NX = G since X is maximal in G. In this situation, Y = ND and 
we claim that D is a maximal subgroup of X. To see this, suppose D < U < X. Then 
YU = NU is a subgroup and Y < YU < G since YUC\X = U, and this contradicts the 
maximality of Y in G. 

Since D is maximal and of 7r-index in X, we know from Lemma 2.1(b) that 6 {X -. D ) 
is the permutation sign character of the action of D on the right cosets of D in X. This 
action, however, is permutation isomorphic to the action of D on the right cosets of Y 
in G. (Intersection of a coset of Y with Ogives the desired isomorphism.) Since 8 (G:Y ) is 
the permutation sign character of the action of Y on the right cosets of Y in G, we see 
that (6 (G:Y ))d is the permutation sign character of the action of D on these cosets. In this 
situation, we have {8(g-.y))d = Vd) and assertions (a) and (b) follow. 

We can now assume that M is the core of Y in G and similarly, it is the core of X. 
Working in G/Af, therefore, we see that X/Mand Y/M are core-free maximal subgroups. 
They have 7r-index, and so they contain (V(G/M), which is therefore trivial. It follows 
from Lemma 2.2 that Xj M and Y/M are conjugate, and thus X and Y are conjugate in G. 
This is impossible, however, since XY = G, and this contradiction completes the proof 
of (a) and (b). 

The proof of (c) is now immediate because 

«r c )x = (fen*)*)* = (^g:y)0)d) x = CWx>)* = CM* 

where the penultimate equality follows from (a). ■ 
We turn now to the definition of the set D„(G). If G is 7r-separable, we recall (for 
comparison) that B ff (G) is a superset of the set X„(G) of 7r-special characters. In fact, the 
7r-special characters of G are exactly those members of B n (G) that have 7r-degree and 
every member of B„(G) is induced from an appropriate 7r-special character of a subgroup 
of G. (See [7] for an exposition of this.) Given that 2 £ tt, the set D* (G) is also a superset 
of ^r(G). By the definition given in [4], it consists of those irreducible characters of G 
that can be obtained via 7r-induction from 7r-special characters of subgroups of G. In 
other words, x € Irr(G) lies in D ff (G) precisely when there exists a ?r-special character 9 
of some subgroup U C G such that B* G = \- 

Note that by the transitivity of 7r-induction, it is clear that if// C G and xp e D„(//) 
with V* G = X € Irr(G), then x 6 D 7r (G). This proves the 'if part of Theorem E. 
(Theorem E is fully proved in [4], and we say no more about it here.) 
We can now prove Theorem D. 

Proof of Theorem D. We are given x € D V (G) such that 0 = xw is irreducible 
and our task is to show that xj; G D n (H). By definition of D„(G), we can write x = Q" G for 
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some 7r-special character 0 of some subgroup U C G. Since x is induced from a character 
of U and it restricts irreducibly to //, it follows that UH = G. By Lemma 2.3(c), we 
have \p = \fi = (OuhhY", and we know that this is irreducible. It follows that Outh is 
irreducible, and thus it is 7r-special by Theorem A of [4]. By definition, therefore, tp lies 
in D K (H), as required. ■ 
By Corollary 7.2 of [4], the zr-special characters of G are exactly those members of 
D n (G) that have 7r-degree. For the remaining members of D T (G) we shall need the fol- 
lowing technical result, which is part of Lemma 7.7 of [4]. We quote it here without 
proof. 

LEMMA 2.4. Let G be inseparable, where 2 & ir, and suppose that \ G D n (G). If 
is not a it-number, then there exists N <3 G such that \n has ^-special irreducible 
constituents that are not invariant in G. ■ 

3. Lifting --partial characters. In this section we prove Theorem B and its corol- 
lary. 

Proof of Theorem B. Given <p e 1^(G), we must produce a character x G D X (G) 
such that x° = <P- There exists tp G B ff (G) with tp° = <p and by definition of the set 
B n (G) in [2] (or see the exposition in [7]), there exists a Tr-special character 7 of some 
subgroup W C G such that 1 G = tp. (As explained in [7], we can take the pair to 
be a 'nucleus' for tp.) 

Now let x = 7* G = 0-yf, where 8 = 8 (G:W) is a certain linear character of W such 
that 8 2 = 1 w If x G G is a 7r-element, then since 2 tf- 7r, we know that x has odd order, 
and it follows that 

X (x) = Y G (x) = 1%) = tp{x) = pfx) 

and thus x° = ¥>. Since ifi is an irreducible 7r-partial character, it follows that x G Irr(G) 
and hence by definition, x G D n (G). 

To complete the proof, we show by induction on \G\ that the map x •— » X° » s an 
injection from D*(G) into U(G). Let \ G D W (G) and write <p = \°- We must prove that 
<p is irreducible and that if also £ G D r {G) with £° = </?, then £ = x- We suppose first 
that x(l) is a 7r-number. Then x is 7r-special, and hence x G B n (G). Also, since £° = (p, 
we have £(1) = \{\), and thus £ is also ^-special and lies in B n (G). But we know that 
restriction to 7r-elements defines a bijection from B„(G) to l„(G), and we deduce that 
X = £ and that (p is irreducible, as desired. 

We can now assume that \(1) is not a 7r-number. By Lemma 2.4, there exists a sub- 
group N <3 G such that an irreducible constituent 0 of \n is 7r-special and is not invariant 
in G. Let T < G be the stabilizer of 0 in G and observe that T is also the stabilizer 
of 0°, which is an irreducible 7r-partial character of N. (This is because 0 G B„(N) and 
restriction to 7r-elements defines a bijection from B n (N) to MAO ) 

We have x = oc G for some uniquely determined Clifford correspondent a G lrr(r|0). 
If we write 8 = 8 (G: d and V = 8a, we see that tp nG = a G = x, and since x G D W (G), we 
deduce from Theorem E that tp G D n (T). Clearly, (tp°) G = <p and 0° lies overfl 0 G MA 7 ). 
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By the inductive hypothesis, we know that t/; 0 G M7), and since T is the stabilizer of 9° 
in G, the Clifford correspondence for 7r-partial characters guarantees that tp = (xp°) G is 
irreducible, as desired. (See Theorem 5.3 of [8].) 

Given £ G D*(G) with £° = <p, we know that 9° is a constituent of = (Zn) 0 * and 
so 0° is a constituent of p. 0 , where \i is some (ordinary) irreducible constituent of 
But the irreducible constituents of all lie in D„(A/). (As we mentioned earlier, this is 
Theorem 7.10 of [4].) However, N C T < G and the inductive hypothesis guarantees 
that restriction to 7r-elements defines a bijection from D^(N) to h(N). Since both 9 and 
p lie in D T (AT) and 9° is known to be a constituent of /x°, it follows that 9 = p, and thus 
0 lies under £ and we let 0 G Irr(7/|0) be the Clifford correspondent of £ with respect 
to 9. Thus 0° = £ and we have 77"° = £, where 77 = 60. Reasoning as in the previous 
paragraph, we deduce that n eD n (T) and that if is the Clifford correspondent of <p with 
respect to 9°. It follows that rf* = xp°, and thus 7/ = ip by the inductive hypothesis applied 
to r. Thus £ = if 0 = ^ = x. and the proof is complete. ■ 

Proof of Corollary C. Given that \G\ is odd, we claim that B^(C7) C D W (C7). If 
X 6 B n (G), we know that x = 7 G , where 7 is a 7r-special character of some subgroup 
W C G. But b(G:W) = \w since |JP| is odd; thus x = 1* G , and hence x G D^(C7). Thus 
B*(C7) C D ff (C7) as claimed. But by Theorem B, we have |D„(G)| = (^(G)! = |B„((7)|, 
and hence &*(G) = D*(G), as desired. ■ 

4. Supermonomial partial characters. In this section, we prove Theorem F. The 
principal nontrivial fact on which we rely is that forp-solvable groups with p^2, mono- 
mial irreducible characters of /7-power degree are necessarily supermonomial. This re- 
sult, which is discussed in [8] and proved there as Theorem 10.1, is a relatively easy 
consequence of a deep result of E. C. Dade. The argument we use here to obtain Theo- 
rem F from Theorem 10. 1 is really the same as the proof of Theorem 10.2 in [8], which 
is essentially the odd-order case of Theorem F. 

We need the following easy consequence of Theorem B. 

Lemma 4. 1 . Let H C G, where G is it-separable and 2 -n. Let \ G D T (C7) and 
suppose that x° = a G , where a G I*(//). Then \ = if for some character i] G Irr(//) 
such that rj° = a. 

Proof. To find 77, first choose ip G D n (H) with \p° = a. (This is possible by Theo- 
rem B). Now (xp vG )° = a G = x° is irreducible, and thus xp* G lies in D*(C7) by the easy 
part of Theorem E. Since also x € O n (G) and the map £ ■ — ► £° is injectivc on D w ((7), we 
deduce that \J) nG = \. We can now take 7; = S(c : h)^ to complete the proof. ■ 

Proof of Theorem F. We are given tp g \ p (G), where G is p-solvable and p±2. 
It is clear that if any lift of ip in Irr(G) is monomial, then tp must be monomial. We may 
assume, therefore, that ip is monomial, and our task is to show that tp is supermonomial 
and that its lift in D P (G) is also supermonomial. 
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Write ip = a 6 , where a is a linear /^-partial character of some subgroup A C G, and 
assume that also <p = j3 G , where 8 G l p (B) for some subgroup fi C G. We must show 
that 8 is monomial and we consider first, the case where ip(l) is a power of/?. 

We have x € D p ((7) with x° = <fi. Since a G = <p = (F, it follows by Lemma 4.1 
that we can find i/> € Irr(/f) and // G Irr(Z?) such that i/> c = X = *7 C » where ip° = a and 
tj° = 0. In particular, i/;(l) = or(l) = I, and hence x is a monomial character. Since 
X has p-power degree, it follows by Theorem 10.1 of [8] that it is supermonomial, and 
hence i] is monomial. We deduce that 0 = t/° is monomial, as required. 

In the general case, where </?( 1 ) may not be a p-power, we can assume that (3 is prim- 
itive. By Corollary 5.5 of [8], we know that (3(1) is a power of /? and, of course a(l) = 1 
is also a power of p. In other words, in the language of Section 6 of [8], each of the pairs 
(A, a) and (B,{3) is a p-inducing pair belonging to <p, and thus each of these pairs is a 
node of the graph Q(y). By Theorem 6.2 of [8], therefore, if we replace the pair (A, a) 
by a suitable conjugate pair, we can assume that the two nodes (A, a) and (B, 8) lie in the 
same connected component of this graph. 

Recall that by the definition of the graph, nodes (U, fi) and (V, u) are joined in Q(ip) 
if either U< Pand// = vox V< V and u u = p. We claim that whenever nodes (U,p) 
and (V,i/) are joined, if either of the p-partial characters n or v is monomial, then the 
other is monomial too. To see this, assume that /x is monomial. In the case where U < V 
and \i v = v y it is clear that v is monomial, and so we need consider only the situation 
where V < U and u u = p. To prove that v is monomial in this case, it suffices to show 
that the monomial /'-partial character n of i : is actually supermonomial. But //( I ) is a 
/?-power (since (U, n) was assumed to be a p-inducing pair for y?) and this is the case of 
the theorem we have already proved. Thus u is monomial, as claimed. 

Recall that the pairs (A, a) and (B y (3) lie in the same component of G(p). Since a 
is linear, it is certainly monomial, and thus by the result of the previous paragraph, 8 is 
monomial too, as required. 

Finally, we must prove that x is supermonomial, where x G Dp(G) and x° = V 3 - 
Assuming that x = i> G , where ip € Irr(//) and H C G, we need to show that x{j is 
monomial. If we write 8 = tp°, we see that 8 G = tp and since <p is supermonomial, we 
know that 8 is monomial. 

Now write n = 6 (G: //)V>, so that if 0 = \ and 77 0 = tp° = 13. Then r/ € D P (H) by 
Theorem E, and since 8 is monomial, it follows by Lemma 4. 1 that 77 is monomial. We 
can thus write 77 = X H for some linear character A G Irr(X), where K C H, and we see 
that xp = ((6( C H ))k\) H is monomial, as desired. ■ 

5. One-prime M-groups. Fix an odd prime p. We shall say that a p-solvable group 
G is a DpM-group if every character x € D p (G) is monomial. Of course, an M-group is 
automatically a D p M-group for every odd prime p. (Note also that every group of order 
not divisible by p is a D p M-group since D P (G) consists just of the principal character in 
this case.) 
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One of the principal outstanding conjectures about M-groups is that normal (and hence 
also subnormal) subgroups of odd-order M-groups must also be M-groups. We prove the 
corresponding assertion for D^M-groups, and from this we obtain Theorem A of the 
introduction. 

THEOREM 5.1. I/N <w G, where G is a D p M-group, then N is also a D p M-group. 

It follows that subnormal subgroups of M-groups must be D p M-groups for all odd 
primes p. This imposes a severe restriction on the collection of groups that can be sub- 
normally embedded in an M-group, but it does not settle the normal-subgroup conjecture 
since there do exist odd-order groups that are D^M-groups for all odd primes p, and yet 
are not M-groups. 

Theorem 5. 1 is an easy consequence of the following stronger result. 

THEOREM 5.2. Let N <W G, where G is p-solvable for some odd prime p.Ifx^ 
D P (G) is monomial, then every irreducible constituent of xn is monomial. 

Proof. Since all irreducible constituents of xs he in D P (S) for all subnormal sub- 
groups S of G, it suffices to prove the result in the case where N is a maximal normal 
subgroup of G. We assume this, therefore, and we let ^ be an irreducible constituent of 

X/V. 

Let 7/ be the stabilizer in G of $ and let n G \ir(T | xp) be the Clifford correspondent 
of x with respect to \p. Then rf = x, and by Theorem F, x is supermonomial (since it 
is monomial). It follows that j/ is monomial and we can write rj = X G for some linear 
character A of some subgroup ofHCT. If we can show that i} N = tp, it will follow that 
$ = (-WwA as required. 

We know that t)n = where e is the multiplicity of xp as a constituent of xn- Our 
task, therefore, is to show that this multiplicity is equal to 1. If \G : N\ = p, this is clear, 
and thus we can assume that \G : N\ is not divisible by p. Write (f = x° and 6 = xp°, and 
note that both (f and 9 are irreducible p-partial characters. Also, the multiplicity of 0 as 
a constituent of <ps is 1 since p does not divide \G : N\. (See, for example, Corollary 5.1 
of [8].) It follows that the multiplicity of ip as a constituent of xn cannot exceed 1, and 
the proof is complete. ■ 

Proof of Theorem 5.1. Let 1^ € D P (N). To show that if) is monomial, it suffices 
by Theorem 5.2 to find a character x G D P (G) that lies over xp. Now 0 = xp° lies in l p (N) 
and it is easy to see that there exists <p € lp(G) such that 9 is a constituent of (This 
is Lemma 4.3 of [8].) By Theorem B, choose x £ D P (G) such that x° = <P and observe 
that since N is subnormal, all irreducible constituents of xn lie in D p (A/). Since 9 is a 
constituent of (xn)°, it follows by Theorem B that xj) must be a constituent of xn , and the 
proof is complete. ■ 

Recall that a primitive irreducible character of a 7r-separable group can be factored as 
a product of a ?r-special character and a Tr'-special character. (This theorem was proved 
first in [1], and it is discussed in [7], where it appears as Theorem 2.4.) We can use this 
factorization of primitive characters to prove the following extension of Theorem A. 
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COROLLARY 5.3. Let N <w G, where G is a D p M-group for some odd prime p. If 
V> € Irr(A0 is primitive, then is not divisible by p. 

Proof. Factor iff = a0 y where a is p-special and (3 is //-special. Since a is p-special, 
it lies in D P (N), and hence it is monomial by Theorem 5. 1 . But a is a factor of a primitive 
character, and thus or is primitive, and hence is linear. We conclude that t/;(l) = f3(\) is 
not divisible by p, as required. ■ 

PROOF OF THEOREM A. We are given that N « G, where G is an M-group, and 
thus N is a D^M-group for all odd primes p. By Corollary 5.3, the degree of a primitive 
irreducible character of N can be divisible by no odd prime, and thus it is a power of 2. ■ 

6. Examples and further remarks. We mentioned that in general, the function 
B^( ) does not respect restriction to arbitrary subgroups. If \G\ is odd, however, and 
X € B T (G), then by Corollary 8.4 of [8], there is always a constituent of \h in B„(//) for 
subgroups H C G. In particular, if we know that \h is irreducible, then it lies in B n (H). 
(This is the result that Theorem D generalizes.) If \G\ is even, however, then it is possible 
to have x £ B T (G) with x« irreducible, but with \h & B w (//). This can even happen if 
X is 7r-special, as is shown in Example 8.2 of [4]. If x is 7r-special and 2 ^ 7r, however, 
no such example can occur. We have been unable to decide the following, however. 

QUESTION 6. 1 . Let G be 7r-separable, where 2 ^ 7r, and suppose x € B^(G). Sup- 
pose H C G and that \h is irreducible. Must we have \h € B„(//) in this case? 

Of course, if it were always true that B n (G) = D n (G) for 7r-separable groups G with 
2 & 7r, then Question 6. 1 would have an affirmative answer, by Theorem D. In fact, 
however, B W (C7) and D^G) are different, in general. 

EXAMPLE 6.2. Let ?r = {3}. Then there exists a 7r-separable group G for which 
B,(G) / D,(G). 

Proof. Let G be the semidirect product of the symmetric group on four symbols, 
acting on an elementary abelian group V of order 3 4 , where the symmetric group per- 
mutes a basis {vi , v 2 , v 3 , v 4 } for V in its natural action. Let A be the unique linear char- 
acter of V such that A(v,) = A(v2> = 1, A(v 3 ) = uj and A(v 4 ) = u), where a; is a fixed 
primitive cube root of unity. 

The stabilizer of A in S4 is exactly the transposition t — ( 1 , 2), and thus V has index 
2 in the stabilizer T of A in G. We see that A has exactly two extensions a and (3 to T, 
and these are distinguished by the values a(t) = 1 and 0(t) as —I. Write oP = \ and 
/3 G = V and observe that by the Clifford correspondence, these are distinct irreducible 
characters of G. 

We see that a 0 = /3°, and thus x° = V>° € U(G) by the Clifford correspondence 
for partial characters. Since x ^ V and each is a lift of the same 7r-partial character, it 
suffices to show that x G B ff (G) and rp 6 D W (G). 

First, we consider x- It is easy to check that the pair (K, A) is maximal among tc- 
factored subnormal pairs in G. (See Section 4 of [7] for the relevant definitions.) Since 
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T is the stabilizer of this pair and a e lrr(T) is 7r-special and lies over A, we deduce that 
X = oc G lies in B T (G), as claimed. 

To see that i/> € D n (G), it suffices to show that a* G = tp, and for this purpose, we check 
that a8(G\T) = P- It suffices, therefore, to verify that 6(G:T){t) = —1. To compute fyc.T), 
we choose a subgroup P such that T C P C G, where |G : P| = 3. Then |P : 7*1 = 4, and 
hence 8(pj) = lr and <S(G:r)(0 = f>(GP)(0 by Lemma 2.1(a) and (c). Furthermore, since 
P is maximal in G, we see by Lemma 2. 1(b) that 5(C:P) is the permutation sign character 
of the action of P on the three right cosets of P in G. Since / is not in the core of P, the 
action of / is nontrivial, and thus it is an odd permutation. It follows that 6 (G: p)(t) = - 1 , 
as desired. ■ 

It is amusing to note that in general, the symmetric difference of the sets B n (G) and 
D W (G) consists of characters with degrees that are neither 7r-numbers nor V -numbers. 
That a character with 7r-degree in either set must be in both is a consequence of the fact 
that such a character must be 7r-special. On the other hand, if \ G B T (G), then we know 
that x = 1 G for some Tr-special character 7 of a subgroup W C G. If \ has Tr'-degree, 
then W must have 7r / -index, and thus 8 (G :W) is trivial and \ € D*(G). Our assertion now 
follows (using Theorem B). 

Finally, we consider the class of odd-order groups that can occur as normal (or sub- 
normal) subgroups of M-groups. We know by Theorem 5. 1 that such a group must be a 
DpM-group for every odd prime p. If it were true that only M-groups had this property, 
this would prove the conjecture that odd-order normal subgroups of M-groups must be 
M-groups. Unfortunately, we have the following. 

Example 6.3. There exists an odd-order group G of that is a D p M-group for every 
odd prime p, but that is not an M-group. 

In feet, it is relatively easy to construct groups that are D^M-groups for all odd primes. 

LEMMA 6.4. Suppose that N < G, where N is a normal abelian Sylow q-subgroup 
of G and G/N is a DpM-group for all odd primes p. Then G is a D p M-group for all odd 
primes p. 

PROOF. Let x € D P (G). Then the irreducible constituents ofxN lie in D P (M), and so 
ifp^q, these constituents are trivial and N C kerx- In this case, we can view x as a 
character of G/N, and it is easy to see that \ 6 D P (G/N). (We omit the details of this 
routine argument.) Ifp^q, therefore, x is monomial as a character of G/N and hence 
also as a character of G. 

We can suppose now that p = q so that x = ^° for some ^-special character 9 of 
some subgroup H C G. Since 6 is ^-special, however, it restricts irreducibly to a Sylow 
^-subgroup of//, which is abelian. It follows that 0 is linear and thus x = 0 {G .H)0) G is 
monomial, as required. ■ 

Proof of Example 6.3. Choose any odd-order M-group H that has a subgroup K 
that is not an M-group. (Examples of this abound.) Next, choose an odd prime q not 
dividing \H\ and let N be an elementary abelian <?-group of order q\" :K l Let H act on N 
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by permuting a basis so that K is the full stabilizer of one of the basis vectors and let G 
be the semi-direct product of N by H. 

Since G/N is an M-group, it is certainly a D^M-group for all odd primes p. By 
Lemma 6.4, therefore, G is a D^M-group for all odd primes p, and it suffices to show 
that G is not an M-group. 

There exists a linear character A of N whose stabilizer in H is exactly K, and so its 
stabilizer in G is T = NK. Since A extends to T and T/N is not an M-group, it is easy 
to see that T has a non-monomial irreducible character 9 lying over A. Also, x = 9° is 
irreducible by the Clifford correspondence, and we claim that x is not monomial. 

Suppose that x = ct G for some linear character a of a subgroup X of G. We see that 
\G : X\ = x(l) is not divisible by q, and hence N C X. Since a N lies under x, it is G- 
conjugate to A, and so we can replace the pair (X, a) by a conjugate pair and suppose that 
a N = A. Then X C T and of G ln(T) lies over A and under x- It follows that of = 9, 
and this is a contradiction since 9 is not monomial. ■ 

We close with something only marginally related to the topic of this paper. In The- 
orem E of [4], it was shown that if H C G, where G is 7r-separable and 2 £ tt, and if 
t/> G Irr(fT) is 7r-special and extends to G, then in fact, tp has a 7r-special extension to G. 
It was asked in [4] whether or not this result might remain valid without the assumption 
that 2 ^ 7r. In fact, the answer is 'yes', and even more is true, both in the case where 
2 G 7r and where 2 & tt. 

COROLLARY 6.5. Let HQG, where G is it-separable, and suppose that 9 G I*(//) 
extends to <p G l n (G). 

(a) If 2 G tt. then the lift of 9 in B„(//) extends to the lift ofip in B„(G). 

(b) If! (£ tt, then the lift of 9 in D*(H) extends to the lift ofv in D„(C7). 

PROOF. Suppose first that 2 £ tt and let x € B n (G) with x° = V- Write ib = \h 
and observe that i/> 0 = 9, so that tp is irreducible. By results of [6], there is a 'magic' 
field automorphism that fixes x, and so it also fixes 0 = Xh- As (xp)° = 9 is irreducible, 
Lemma 3.3 of [6] guarantees that ip G B n (H), as required. 

Assuming now that 2 £ tt, let x G D W (G) lift <p and again write t/» = \ H . Then \p° = 9, 
and so xp is irreducible, and it follows that xp G D*(//) by Theorem D. ■ 

COROLLARY 6.6. Let H be ir-separable and suppose 9 G U(H). Then 9 has a lift 
V> G \tt(H) such that whenever H C G, where G is ir-separable, tp extends to a character 
of G iff 9 extends to a TT-partial character of G. 

Proof. If 2 G tt, take ip G B„(//) and otherwise, take xp G D n (H). ■ 
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GREENBERG'S THEOREM FOR QUASICONVEX 
SUBGROUPS OF WORD HYPERBOLIC GROUPS 

ILYA KAPOVICH AND HAMISH SHORT 



Abstract. Analogues of a theorem of Grcenberg about finitely generated sub- 
groups of free groups are proved for quasiconvex subgroups of word hyperbolic groups. 
It is shown that a quasiconvex subgroup of a word hyperbolic group is a finite index 
subgroup of only finitely many other subgroups. 

1. Introduction. 

Introductory remarks. There has been much interest recently in reintroducing geometric 
ideas into group theory. One of the most interesting examples is Gromov's work on the 
class of word hyperbolic groups [Gr]. The notion of a word hyperbolic group arose in the 
work of J. Cannon [C] and M. Gromov [Gr] as a generalization of certain group theoretic 
properties of discrete groups of isometries of classical hyperbolic spaces H", the origins 
of which are to be found in Dehn's solution of the word and conjugacy problems for sur- 
face groups (see [De]). Word hyperbolic groups are finitely presented, and examples are 
provided by finite groups, finitely generated free groups, fundamental groups of compact 
surfaces (except for the torus and the Klein Bottle), and groups which act properly dis- 
continuously and cocompactly on hyperbolic space of any dimension. Finite extensions 
of word hyperbolic groups, and free products of finitely many word hyperbolic groups 
are also word hyperbolic, though a direct product of two infinite word hyperbolic groups 
is not word hyperbolic. We will give some background information about word hyper- 
bolic groups in Sections 2 and 3 and refer the reader to Gromov's original article [Gr], 
and the several commentaries that now exist (for instance [ABC], [Bol], [CDP], [GH]) 
for further information, and proofs of the basic results. 

The object of this note is to prove for word hyperbolic groups analogues of a theo- 
rem of Greenberg for finitely generated subgroups of Fuchsian groups (see Theorem 1 , 
Part (2) below). In our case these results concern quasiconvex subgroups (for the full 
definition see Section 2), which are a special kind of finitely generated subgroup. In- 
deed, in a free group a subgroup is finitely generated if and only if it is quasiconvex. In 
a word hyperbolic group, all finite subgroups, all cyclic subgroups, and all subgroups of 
finite index are quasiconvex. Also, a quasiconvex subgroup of a word hyperbolic group 
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is word hyperbolic. It is not true in general that a finitely generated subgroup of a word 
hyperbolic group is word hyperbolic as there exist finitely generated subgroups of word 
hyperbolic groups which are not finitely related (see for instance [R], [BMS]). Moreover, 
recently N. Brady [Br] constructed a remarkable example of a torsion-free hyperbolic 
group which contains a finitely presented subgroup which is not itself word hyperbolic. 

An important part of Gromov's work involves the introduction of a "boundary" which 
compactifies the Cayley graph of a word hyperbolic group (in fact it compactifies a 6- 
hyperbolic space). This construction generalizes the usual compactification of w-dimen- 
sional hyperbolic space by a n - 1 dimensional sphere. A similar construction for groups 
was also investigated by Floyd in his thesis (see [F]). 

In Section 2 of this paper we define a very rudimentary form of boundary for the 
group, which is not in fact a compactification. We show that several results about word 
hyperbolic groups and their subgroups can be proved using exactly the same techniques 
which are used in classical hyperbolic geometry, provided certain preparatory work is 
done. We introduce the notions of a boundary of a subgroup of a group, and of the convex 
hull of a subset of the boundary. We then show in 2.4(i) that for a quasiconvex subgroup 
A, the quotient of the convex hull of the boundary of A by the action of A is finite. Using 
this, we obtain: 

THEOREM 1 . Let G be a word hyperbolic group, and let A be an infinite quasiconvex 
subgroup. 

(1) IfB is an infinite quasiconvex subgroup ofG, and A (IB has finite index in A and 
in B, then AH B has finite index in AM B, the subgroup generated by A and B in 
G. 

(2) The subgroup A has finite index in only finitely many distinct subgroups of G. 

(3) The subgroup A has finite index in its virtual normalizer 

VN G (A) = {geG\ [A: A HgAg''] < oo and [gAg~ ] :A HgAg' 1 ] < oo}. 

In Section 3 we study the boundary of a 6-hyperbolic space (for instance the Cayley 
graph of a word hyperbolic group) as defined by Gromov, as a compactification of the 
space. The subgroup structure of word hyperbolic groups is now discussed from the point 
of view of understanding their limit sets in the boundary. 

We improve on the results of Section 2, and show (Proposition 3.4) that an infinite 
subgroup is quasiconvex if and only if the quotient of a convex hull (appropriately de- 
fined) of its limit set by the action of the subgroup has finite diameter. 

It is not possible to use the standard notion of convexity and convex hulls when work- 
ing with word hyperbolic groups. Indeed, as it was demonstrated in a remarkable example 
of M. Bridson and G. Swarup [BS], it can happen that a (classical) convex hull of a fi- 
nite subset of a Cayley graph of a group coincides with the whole group. Thus certain 
adjustments have to be made and an appropriate substitution for the classical notion of 
convex hull should be used. 
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Our new characterization of quasiconvex subgroups of word hyperbolic groups 
stresses the similarity between quasiconvex ity and geometrical finiteness for classical 
hyperbolic groups and immediately yields the following result of G.Swarup [Swa]. 

THEOREM 2. Let G be a torsion-free geometrically finite group of isometries ofU" 
without parabolics. Then G is word hyperbolic and a subgroup AofG is quasiconvex in 
G if and only if it is geometrically finite. 

We also obtain a criterion of commensurability for quasiconvex subgroups of word 
hyperbolic groups in terms of limit sets (Lemma 3.8) and obtain another proof of Theo- 
rem 1 as well as 

THEOREM 3 . Let A be an infinite quasiconvex subgroup of a word hyperbolic group 
G and let B be a subgroup ofG containing A. Then A has finite index in B if and only if 
A contains an infinite subgroup C which is normal in B. 

In Section 4 we remark that it is possible to give a criterion of quasiconvex ity similar 
to Proposition 3.4 in terms of the elementary boundary we use in Section 1. 

Theorem 3 has also been obtained by Mihalik and Towle [MT] and results similar 
to those of Section 3 were independently obtained by E. Swenson [Swe] and R. Gitik, 
M. Mitra, E. Rips and M. Sageev [GMRS]. Parts (1) and (2) were originally proved 
for finitely generated subgroups of free groups by L. Greenberg [G], using the usual 
compactification of the hyperbolic plane by a circle. He showed that part of the boundary 
can be naturally associated to a subgroup A of a discrete subgroup G of PSL(2, R). He 
then showed that two subgroups have the same associated boundary if and only if they 
arc finite extensions of a common subgroup. 

We would also like to bring to the reader's attention Stallings' beautiful paper "The 
topology of finite graphs" [St] where most of Theorem 1 is proved for finitely generated 
subgroups of free groups. 

Some comments on Theorem 1. Before proceeding to the definitions and proofs of our 
main theorem, we give some examples to illustrate the necessity of some conditions 
concerning infiniteness and quasiconvexity. Notice that a finite subgroup is always qua- 
siconvex. 

Statement (3), may fail if H is finite or if H is infinite but not quasiconvex. Consider the 
group Q = (a, b \ [a, b] 2 = 1 ) and H = { 1 }. Then Q is word hyperbolic (being a small 
cancellation group) and there are infinitely many distinct conjugates of the subgroup 
hi = gp([a, b]) = T-i of Q (by the usual small cancellation arguments). Thus H is a 
subgroup of finite index in infinitely many subgroups of G although H is quasiconvex in 

Q. 

When G contains an infinite cyclic subgroup, the trivial subgroup H = { 1 } is con- 
tained in infinitely many cyclic subgroups (each of which is quasiconvex when G is word 
hyperbolic). Concerning Theorem 3, there is a word hyperbolic group G which has a nor- 
mal subgroup H isomorphic to the fundamental group of a closed hyperbolic surface with 
the quotient group Greenberg's theorem for quasiconvex subgroups of word hyperbolic 
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groups G/H isomorphic to Z. Again, in this case, the normalizer of H is all of G, and 

\G:H\= oo. 

One can also construct an example of an infinite, finitely generated (but not quasicon- 
vex) subgroup H of a word hyperbolic group G which has finite index in infinitely many 
subgroups of G, as follows. Let Q be the one-relator group with torsion defined above. 
Now apply the construction of E. Rips [R] to obtain a short exact sequence 

\^K-*G->Q^ 1 

where G is a torsion-free word hyperbolic group (as it satisfies a small cancellation con- 
dition), and K is an infinite, normal two-generator subgroup of G. Let q be a conjugate 
of [a, b] in Q and g be an element in the preimage of q in G. Put K q = gp(K,g). Then 
g 2 6 K, g g K and K is a subgroup of index 2 in K q . Moreover, the image of K q in Q is 
the cyclic group of order two gp(q). Thus for different conjugates q of g we get different 
subgroups K q of G containing AT as a subgroup of finite index. 

Hyperbolic groups with all finitely generated subgroups quasiconvex. We say that a 
word hyperbolic group G belongs to class (0 when a subgroup A of G is quasiconvex 
in G if and only if A is finitely generated. 

Among the groups known to belong to the class (0 are finitely generated free groups 
and fundamental groups of closed hyperbolic surfaces (see [Sho] and [Pi]). It is also clear 
that finite extensions and subgroups of finite index (in fact all quasiconvex subgroups) of 
groups in (0 belong to (0. G. Swarup has shown [Swa] that if G is a geometrically finite 
torsion-free Kleinian group without parabolics with a nonempty discontinuity domain 
(i.e. the limit set of G is not the whole sphere S 2 ) then G is in (0. The last observation 
is based on a result of W. Thurston [Mo]. 

Notice that any group G in the class (0 has the Howson property, that is the intersec- 
tion of any two finitely generated subgroups is finitely generated (indeed, the intersection 
of two quasiconvex subgroups is quasiconvex and so is finitely generated, finitely pre- 
sentable and even word hyperbolic). This approach to Howson property is discussed in 
more detail in [Sho]. 

In the class (Q), Theorem 1 can be stated in terms of finitely generated subgroups. In 
[Ba] H. Bass proved Theorem 1 (2) in these terms for the case when G is the fundamental 
group of a finite graph of groups Y with finite vertex groups. Such a group G lies in 
(Q) since it is virtually free. There are two natural proper hyperbolic spaces on which G 
admits a properly discontinuous cocompact isometric action. One is the Cayley graph of 
G with respect to some finite generating set. Another is the Bass-Serre universal covering 
tree for the graph of groups Y. The result can be established for G by studying either of 
these actions. The original proof of H. Bass employed the second action but the ideas 
involved in it are quite similar to ours. 

In the case when G is a free group of finite rank, the Bass-Serre tree coincides with 
the Cayley graph of G with respect to a free basis. This action was studied by J. Stallings 
in [Sta] to obtain the same statement for free groups. The original proof of this result 
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by L. Greenberg (which also applies to hyperbolic surface groups) used the fact that a 
free group G admits a discrete properly discontinuous geometrically finite action in H 2 
and therefore the action of G on the (classical) convex hull of its limit set is properly 
discontinuous and cocompact. 

From our point of view the proof given in the present note captures the similarity of 
all these approaches. 

2. Definitions and Proof of Theorem 1. 

Some definitions. If (X, d) is a metric space and / is a connected subset of a real line, a 
map/: / -» X is called geodesic if d(f{t\f{s)) = |/ - s| for any s, t e /. Sometimes the 
image of this map is also termed a geodesic. A metric space is termed geodesic if there 
is at least one geodesic joining each pair of points. 

If A is a geodesic triangle (that is a triangle with geodesic sides) in a metric space (X, d) 
with vertices x,y, z and geodesic sides [x,y], [x,y], [y, z] then there are unique points s, q, r 
on the sides [x,y], [x, z] and [y, z] such that d(x, s) = d(x, q), d{z, q) = d(z, r) and d(y, s) = 
d(y, r). The points s, q, r are called the vertices of the inscribed triangle in the triangle A. 
The triangle A is said to be 8-slim if, for any points s' on [x, y] and q' on [x,z] chosen 
such that d(x,s') = d(x,q') < d(x,s) = d(x,q), we have d(s',q') < 8 (and the analogous 
condition holds interchanging x, y and z). 

A geodesic metric space (X, d) is called 8-hyperbolic if all geodesic triangles are 8- 
slim. 

Let G be a group generated by a finite set S. We define the S-length of an element 
g e G, denoted by l s (g), as the minimal number / such that g can be expressed in the 
form g ss si • • • s, where s, e 1 . A minimizing expression is termed an S-geodesic rep- 
resentative of g. We will often confuse formal words in the alphabet S with the elements 
they represent in G. 

The empty word e by convention represents the identity element of G. Define 
ds(g\, gi) = lsigi l gi\g\,g2 € G. This defines a metric on G called the word metric 
associated to S. The word metric ds can be extended to a metric (also denoted ds) on the 
Cayley graph T(G, S), by declaring each edge to be isometric to the unit interval. This 
metric is clearly invariant under the natural left action of G. 

A finitely generated group G is called word hyperbolic if for some finite generating 
set S of G there is 8 > 0 such that all geodesic triangles in the Cayley graph (r(G, S), ds) 
of G are 6-slim. (It turns out that this property is independent of the finite generating set 
chosen — see [Gr], [ABC], etc.). 

In general, a subset /I of a geodesic metric space (X, d) is termed c-quasiconvex in A' if 
any geodesic [a\ , a 2 ] in Ajoining points a\ , a 2 e A is contained in an e -neighbourhood of 
A. IfGisa finitely generated group and A is a subgroup of G, we say that A is quasiconvex 
in G if for some finite generating set S of G A is a quasiconvex subset of the Cayley graph 
T(G,5). 

The following properties of quasiconvex subgroups and word hyperbolic groups and 
their quasiconvex subgroups are of importance to us here: 
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PROPOSITION 2.0. Let Gbea finitely generated group, and S a finite set of generators 
for G. 

(1) [GS] If A is a quasiconvex subgroup of G then A is finitely generated. 

(2) [GS] If A is a quasiconvex subgroup of G then for any finite generating set T of 
A and for any finite generating set S of G there is a constant B > 0 such that 
dT(a\,ai) < Bds(ai,a 2 ),a\ t a 2 € A. 

(3) [ShoJ If A\,...,Ak are quasiconvex subgroups of G, then A = A \ D • • • PI A k is 
quasiconvex in G. 

If in addition, G is a word hyperbolic group, then: 

(4) [BGSS, Theorem 3.1] If A is a finitely generated subgroup of G and for some finite 
generating set T of A and some finite generating set SofG there is a constant B > 
0 such that drip \,a{) < Bds(a\,a2),a\,a2 € A, then A is a quasiconvex subset of 
T(G, S). (Together with (2) this implies that the definition of quasiconvexity for a 
subgroup of a word hyperbolic group G does not depend on the choice of a finite 
generating set of G). 

(5) [CDP, Chapter 3, Proposition 4.1] If A is a quasiconvex subgroup of G then for 
any finite generating set T of A and for any finite generating set S of G A is a 
quasiconvex subset ofY(G, S). 

(6) [ABC, Lemma 3.8] If A is a quasiconvex subgroup of G then A is also word 
hyperbolic. 

(7) If A, B are subgroups of G such that A has finite index in B, then A is quasiconvex 
in G if and only ifB is quasiconvex in G. 

(8) [GH, Chapter 4, Theorem 13] There are only finitely many conjugacy classes of 
elements of finite order in G. 

(9) [GH, Chapter 8, Theorem 37] An infinite subgroup of G contains an element of 
infinite order. 

(10) [ABC, Corollary 3.4] If C is a virtually cyclic subgroup of G then C is quasiconvex 
in G. 

(11) [N] If <j> e Aut(G) and A is a subgroup of G, then A is quasiconvex in G if and 
only if <f>(A) is quasiconvex in G. 

Other interesting properties of word hyperbolic groups, for instance that G is finitely 
presented and has a linear isoperimetric function, will not be needed here. For proof of 
these results and further information about quasiconvexity and word hyperbolic groups, 
we refer the reader to [Sho], [Gr], [ABC], [GH] and the other hyperbolic references. 

An elementary boundary for a finitely generated group. In this section G is a finitely 
generated group, and S is a finite set of generators. The group G acts by left multiplication 
on the Cayley graph r s (G\ by isometries. We construct a 'boundary' for G, which is 
essentially the boundary defined by Gromov for word hyperbolic groups deprived of its 
topology. 

Consider the set of subsemigroups © = { {g 1 \ i > 0},g € G) = def { \g£G). 
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We now extend to a larger set (where there is an obvious left action induced by left 
multiplication), and define I=GxS = A,g GGj = {hg+,h,g 6 G}. 

We define an equivalence relation on X by: /ig + ~ A'g 7 if {/ig*;/ > 0} lies in a 
bounded neighbourhood of {h'g^'J > 0} and {h'g ,J ;j > 0} lies in a bounded neighbour- 
hood of {hg\i > 0}. 

This is clearly an equivalence relation, and we denote the equivalence class of hg + by 
[hg + ] (and [hg] = [h(g~ l T]). Notice that [g + ] = [g^] if and only if g and g 7 have a 
common (positive) power: there is some word d and infinitely many pairs (/J) of positive 
numbers such that gf = g'U. 

The elementary boundary of G is defined to be dG = (X/ ~) - {[!]}. 

We remove the equivalence class of the identity element due to its special nature — it 
clearly contains all elements of finite order. One consequence of this is that a torsion 
group will have empty boundary. 

It is clear from the construction that G acts on dG on the left by h ■ [h'g+] = [hh'g*]. 

LEMMA 2.1. (i) If\g + ] = [{g~] then g has finite order in G, and [g + ) = [l ]; 

(ii) The left action of G on dG by left multiplication is the same as the action induced 

by conjugation h ■ [h'g + ] = [{hh' h-'){hgh^n 
(Hi) The definition ofdG is independent of the choice of finite generating set. 

Proof, (i) If the positive powers of G lie in a bounded neighbourhood of the neg- 
ative powers of g, then there is some element d inG such that g* = g~Jd, andg* = g~ l d 
for some ij, k,l>0. It follows that d = = g* 1 in the group G. As this happens for 
infinitely many such quadruples, g has finite order. 

(ii) Notice that d^h 1 , hg 1 ) = ls(h), so that {(hh'h~ l ) hg i = hh'g 1 \ i > 0} lies in 
a ^(A)-neighbourhood of {(M'/T ') • hg i h~ i = hh'gh'* \ i > 0} and vice versa. 

(iii) Choosing a finite set of generators Y for G changes length in the group by at most 
a constant factor, so that equivalence is unchanged. 

Notice that as a result of part (ii) of this lemma, in each equivalence class there is 
a representative of the form {g + }, i.e. Z/ ~= ©/ ~. We now study subsets of this 
boundary which correspond to subgroups. 

Definition. Let A be a subset of G. We define the boundary of A in G, to be: dcA = 
{[a'] | a € A}. Notice that, by (ii) above, <5gG = dG. 

For a subset Y of the boundary dG, we define the stabilizer of Y to be: 

Stab c (y)={gGG|gK= Y}. 

For a fixed finite set 5 of generators for G, define the convex hull of Y to be: 9is(Y) = 
{v e G such that there are g,g\ h,h' G G such that [Ag + ], [h'g l+ ] e Y, and there are two 
infinite, strictly increasing sequences n„ m„ such that v lies on a 5-geodesic from hg"' to 

Notice that, while the definition of the stabilizer is independent of the choice of gen- 
erating set, the convex hull does depend on the choice. We will however refer to Stab(//) 
and ?{(Y) when the ambient group and generating set are clear from the context. 
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EXAMPLES. 0. By Lemma 2. 1 (i) above, the boundary of a finite group, (or indeed 
of an infinite torsion group) is empty. 

1 . The boundary of the infinite cyclic group has two elements, (which we can think of 
as ±oo), and the stabilizer of each of these elements is the entire group. With the single 
generator 1, the convex hull of the boundary is clearly the whole group, as is the case for 
any finite set of generators. 

2. The free abelian group of rank two: each maximal cyclic subgroup has a naturally 
associated "slope", and each rational number J, n > 0 (and 0, oo) defines a cyclic sub- 
group (x^y). The boundary of this cyclic subgroup is two points, and the boundary of 
the group is a countable set (of rational slopes). All rank two subgroups have the same 
boundary, that of the whole group. Notice that the convex hull of the boundary of any 
nontrivial subgroup is the whole group. 

3 . The free group of rank two: each cyclically reduced element g which is not a proper 
power has its own equivalence class [g + ]. The Cayley graph with respect to a set of free 
generators is a tree, and the convex hull of the boundary of a cyclic subgroup generated 
by a cyclically reduced word g is the 'axis' — the line containing the elements of this 
subgroup (i.e. vertices corresponding to initial segments of powers of g). 

For a noncyclically reduced word, the convex hull is the translated axis of the cycli- 
cally reduced part; if w = vuv~ 1 with u cyclically reduced, then the elements of !H(d(w)) 
are the vertices of the Cayley graph corresponding to vu\ where u' is an initial segment 
of the word u m , for m G Z. 

LEMMA 2.2. Let G be a finitely generated group, S a finite set of generators and 
YcdG. 

(i) Stab(y) acts freely on the convex hull tf;(Y). 

(ii) If A is a subgroup of G, A C H G (A) C Stab^)- 

(Hi) If A and B are subgroups of G, and A has finite index in B, then d G A = d G B. 

Proof, (i) G acts freely on itself on the left. It suffices to show that if v G Ms(Y), 
and be Stab(y), then bv G H>(Y). 

Suppose that v lies on geodesies from hg 1 to h'gd, [hg + ] y [h'g >+ ] G Y. For b G G,bv 
lies on a geodesic from bhg* to bh'g 0 ', and for b G Stab(y), it follows that bv G 9fs(X). 

(ii) As in Lemma 2. 1 (ii) above, for a G A,d(ag i ,ag i a~ i ) < \a\,so{ag i } ~ {agfa' 1 }. 
In fact, as long as a normalizes A, this is an element of do A. 

(iii) If b G 5, then b k = a for some a G A, k > 0, so = [a + ]. m 
(Notice that in the proof of (iii) we did not use the fact that k can be chosen to be 

bounded. Thus it was actually proved that if A is a subgroup of B and for each element 
of B some power of it belongs to A, then lb G A = d G B). 

We now concentrate on word hyperbolic groups and their quasiconvex subgroups, 
which have particularly nice properties. Recall that all cyclic subgroups in a word hy- 
perbolic group, are quasiconvex. 
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LEMMA 2.3 . Let A be a quasiconvex infinite cyclic subgroup of the finitely generated 
group G. Then: 

(i) M(!b G A) is infinite; 

(ii) if, in addition, G is a word hyperbolic group, then !tf(d G A) is contained in a 
bounded neighbourhood of A, and is thus quasigeodesic too. 

PROOF. Fix a finite set of generators S for G. 

(1) Let (c) be a K-quasiconvex infinite cyclic subgroup. We show that all geodesies 
from c™ to c~ m pass through a ball of bounded radius about the identity vertex, for all m 
sufficiently large. Notice that these geodesies all lie in a K neighbourhood of the quasi- 
convex subgroup (c). 

Consider a geodesic triangle with vertices \,c m ,c~ m . Let vi,v 2 be adjacent vertices 
on the geodesic a joining c™ and c~ m such that there are positive numbers p,q > 0 
with d(vi,c'P) < K and d{v 2 ,c«) < K. Then d{c~P,^) < 2K + 1; this bounds p + 
q, and in particular bounds p and q. Hence the geodesic a passes through a bounded 
neighbourhood B of the identity element. It follows that there is an infinite sequence 
{m, } and geodesies 7, joining d"' and c~ m ', and a vertex v G B, such that every 7/ passes 
through the vertex v. Thus Hs(d G A) is nonempty, and by 2.2(i) it is therefore infinite. 

(ii) Suppose that v G lHs(d G A). Let {hg*} and {h'g?'} be sequences representing 
elements of d G A such that v lies on infinitely many geodesies joining the points hgf and 
h'gf'. There is a constant C> 0, and a, b G A such that {hgf \ i > 0} and \h'g' j \j > 0} 
are contained in C neighbourhoods of the cyclic subgroups (a) and (b). 

Choose ij sufficiently large so that min{d(y t gf), d(y,g! j )} >26 + C. Let a',b' G A 
be nearest elements of A to hg> and h'gf j . 

By studying the quadrilateral with geodesic sides and vertices a', b\ hg y h'g" \ we see 
that the vertex v is at distance at most 26 from some point on a geodesic joining a' and 
b'. As A is quasigeodesic with constant K, this means that v is at distance at most K + 25 
from ,4. ■ 

COROLLARY 2.4. Let A be a quasiconvex subgroup of the word hyperbolic group G. 

(i) The quotient space Mx{A)/A is finite. 

(ii) A has finite index in Stab0 G A). 

Proof. As 9((A) lies in a bounded ^-neighbourhood of A, and A acts freely on 
9({A), each element of the quotient space has a representative of length at most These 
represent the cosets of A in Stab(&4). ■ 

It is now easy to deduce Theorem 1 : 

Proof of Theorem 1 . (1) If A C\B has finite index in A, and A is quasiconvex in G, 
then AC\ B is also quasiconvex in G. By Lemma 2.2 it follows that d G A = d G (A Pi B) = 
d G B. As A and B both stabilize d G (A n B), so does A V B. Hence AHB < AV B < 
Stab(a G (^ n B)), and applying 2.4(ii) and the quasiconvexity ofAHB gives the result. 

(2) If A has finite index in B, then B G A = d G B, and A < B < Stab@ c A) = C 
If A is quasiconvex then B is a finite index subgroup of C, of index at most [C : A]. 
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There are only finitely many such subgroups. In fact, if { 1, g\ , . . . ,g k } is a set of coset 
representatives for A in C, then B is obtained by adjoining some subset of the {g,} to A. 

(3) Suppose that g e VN G (A). Put U = gAg ] HA. Since (J is of finite index in A 
and in gAg~ 1 , K = dc(U) = c)g(A) = dx(gdg~ 1 ). However dx(gAg~ 1 ) is equal to the set 
{g[a+] | a e A} that is to gK. Thus gK = K and g G Stab G (/Q. Since g e VN G (A) was 
chosen arbitrarily, we conclude that A < VN (; (A) < Stab G (K). By Corollary 2.4(H) this 
implies that A has finite index in VNc(A). 

Notice that part (2) of the theorem gives another way to see that (a, b\b~ l ab = a 2 ) is 
not word hyperbolic. The subgroup (a) has index 2 in (bab~ l ), and index 2" in (b"ab~"). 

3. Gromov boundary and quasiconvexity in hyperbolic groups. 

Definitions and notations. In order to obtain stronger results than those obtained in the 
Section 2, we require a boundary with a topological structure. Such a boundary has been 
defined by Gromov for ^-hyperbolic spaces (see also Floyd's work [F]). For the detailed 
discussion and proofs of the basic properties of this compactification (for instance that the 
elementary boundary of Section 2 can be viewed as a dense subset), the reader is referred 
to [Gr, Chapter 1,3,7], [CDP, Chapter 2], [GH, Chapter 5-7] and [ABC, Chapter 4]. We 
here give some of the basic definitions that we shall need. 

A metric space (X,d) is said to be proper if all closed metric balls in X are compact. 
Any proper 5-hyperbolic metric space (X, d) can be compactified by adding points of 
the ideal boundary (or visual boundary) dX to X. Each point of the boundary dX is an 
equivalence class of geodesic rays r: [0, oo) — » X where rays r t and r 2 are equivalent 
if sup{d(r\(t),r 2 (t))} < oo. It is not hard to see that if n and r 2 are equivalent then 
sup{</(r,(/W[0,oo)) | t € [0,oo)} < max{i/(ri(0),r 2 (0)),2<5}. 

There is another way to think about points of the boundary. Fix a point p € A'and put 

{x,y) p = (l/2)(d(x,p) + d(y,p) - d(x,yj) 

for any x,y € X. Notice that, considering a geodesic triangle with vertices p,x,y, we see 
that (x,y) p is equal to the distance from p to the two vertices of the inscribed triangle 
which lie on the sides [p,x] and [p,y] (see the introduction to Section 2). 
We say that a sequence (x, ), eN of points in X converges to infinity if 

lim{inf(jc,,jc,) p } = oo. 

n—oo ij>n J r 

Two sequences (x,) lgN and (y ( ),gN converging to infinity are said to be equivalent if 

lim { inf (jc„ y.) p } = oo. 



One can check that the last two definitions do not depend on the choice of a basepoint 
p € X. It follows from the definition that if (x,) /eN and (y,) ieN are sequences of points 
in X such that for some finite K, d{x u yi) < K for each i € N then (Xj) ieH converges to 
infinity if and only if (y,), eN converges to infinity, and the sequences are equivalent in 
this case. 
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The points of the ideal boundary dX can be identified with equivalence classes of 
sequences converging to infinity. If a is the equivalence class of (x,) ieN we write 
lim/^KjoX, = a. It is obvious that if r: [0,oo) — ♦ X is a geodesic ray then the sequence 
jc, = r(i) converges to infinity. It is also clear that if r\ , r 2 are equivalent geodesic rays 
then the sequences ( r i(')) l6N and ( r 2(0) ieN are also equivalent. 

Moreover, since (X, d) is proper, any sequence converging to infinity is equivalent to 
the sequence (K0) igN f° r some geodesic ray r. This gives an identification between the 
points of ideal boundary and equivalence classes of sequences converging to infinity [Gr, 
7.5]. 

It can be shown (see [CDP, Chapter 2, Theorem 2. 1 ]) that for any two distinct points 
a y b G dX there exists a (not necessarily unique) bi-infinite geodesic r: (-co, oo) — ► X 
such that lurii-^oK-O = and lim f _ 00 r(j) = b. This geodesic r is sometimes denoted 
by (a, b) and referred to as a geodesic joining a and b. 

Analogously, for any point a G dX and for any point x G X there is a geodesic ray 
r: [0, oo)—*X such that r(0) = x and lim™ r(j) = a. Such a ray r is referred to as a 
geodesic joining x and a and is sometimes denoted [jc, a). 

Fix a basepoint p of X and let r\ , r 2 : [0, oo) — » X be geodesic rays joining p to points 
a and b in the boundary. We put 

(r,,r 2 ) p = sup (r,(0,r 2 (/)) p = lim( ri (t) y r 2 (t)) p . 

/G[0,oo) 

We define (a, b) p to be the infimum of the numbers (ri , r{) p where n , r 2 vary over all 
geodesic rays joining p to a and b. In fact it can be shown that for any such r\ and r 2 , we 

have \(n,r 2 ) p -{a,b) P \ < 105. 

The boundaiydY'ofA r is topologized by putting lim.-^jofl; = a if lim f _ KX) (a < ,a)p = oo, 

where a, a, € dX 

It is easy to describe this topology in the language of sequences converging to infinity. 
Namely, let a ny n = 1 , 2, . . . , be the equivalence class of a sequence (x n/ ), 6N and let a be 
the equivalence class of a sequence (x,), e N. Then linij—^xj a„ = a if 

JimJ mf (x h x mJ )p} -* oo. 

All the definitions given above do not depend on the choice of a basepoint. More- 
over dX and XU dX are compact with the topologies described above [CDP, Chapter 2, 
Proposition 3.2]. 

We need one more piece of notation. If x G Xand r is a geodesic ray joining p with a 
point a G dX, we put (x,r) p = lim(jc,r(0) p = sup{(jc,r(0) p I < € [0,oo)}. One observes 
that if a, 6 are distinct points in dX, and points x € n = [p, a),y G r 2 = [p, b) are chosen 
such that M = {x,b) P = (y,*) P , and </(*,/>) > 50(A/+ 1)5, rf(y,/>) > 100(A/+ 1)5, then 
ix,y) py (x,r 2 ) p , and (r,,.y) p ) are within 105 of (r,,r 2 ) p . 

Convex hulls and limit sets in hyperbolic spaces. Let * be a proper 5-hyperbolic space 
for some 8 > 1 . 

We need the following obvious lemma. 
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LEMMA 3.1. Any ideal geodesic quadrilateral, that is a quadrilateral with vertices 
on the boundary of X, is a 1008-quasiconvex subset ofX. 

Proof. This follows immediately from the fact that ideal quadrilaterals in X are 
106-thin, that is any side is contained in a 106-neighborhood of the union of other sides. 

We can now define the convex hull of a subset of the boundary. We can now use 
infinite rays where in Section 2 we had to make explicit reference to infinite families of 
geodesies. 

DEFINITION, (see [Gr, 7. 5. A]) Let X be a proper 6-hyperbolic space with 8 > 1, and 
let K be a subset of dA' containing at least two points. Then the set CK = U fl ,f>e*( a > b ) is 
called the convex hull of K in X. 

Lemma 3.2. Let X and K be as above. Then CK is 1 006-quasiconvex in X. 

PROOF. Let x,y G CK. Then x G (a,b) and>> G (c,d) for some a,b,c,d G K. 
Assume that the points a, b, c, d are distinct (it will be clear from the argument that the 
case when some of them coincide is completely analogous). 

Join points b and c by a geodesic {b, c) and points d and a by a geodesic (d, a) in X. 
By definition of CK the ideal quadrilateral 

D = (a, b) U (b, c) U (c, d) U (d, a) 

is contained in CK. By Lemma 3.1a geodesic (x,y) is contained in 1006-neighborhood 
of D and so in 1006-neighborhood of CK. Thus CK is 1005-quasiconvex. 

Let (X, d) be as above and suppose a group G acts on A' by isometries so that the action 
is properly discontinuous (i.e. for any compact Q in X the set {g G G | Q PI gQ ^ 0} 
is finite) and cocompact (that is X/G with the quotient topology is compact). The action 
of G on X induces an action of G on dX by homeomorphisms: namely an element g of 
G takes the equivalence class of a geodesic ray r to the equivalence class of the ray gr. 
Equivalently, if (a,), eN is a sequence converging to infinity, then an element g of G takes 
its equivalence class to the equivalence class of the sequence (ga,-),-eN (which obviously 
also converges to infinity). 

Pick a basepoint p G X. For any subgroup A < G define the X-limit set of A, denoted 
by dxiA), as the collection of all those b G dX which are the limits of some sequences of 
points from Ap (in the topology of A'U dX). It is easy to see that this definition does not 
depend on the choice of the basepoint p G Xand that dx(A) is always closed. 

Analogous to the results of Section 2, the limit sets defined above have the following 
properties: 

Lemma 3.3. Let (X, d)bea proper, 8-hyperbolic space, and G a group acting prop- 
erly discontinuously and cocompactly by isometries on X. Let A and B be subgroups of 
G. Then: 

(1) The group G is word hyperbolic. 

(2) d x (A) = 0 if and only if A is finite. 
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(3) If(c) is an infinite cyclic subgroup of G then dx((c)) consists of two distinct points 
c + = lim, .oqC 1 andc — limz-^oC - '. 

(4) If A is contained in B then d x (A) C dx(B). 

(5) If A has finite index in B then dx(A) = dx(B). 

(6) If A is infinite then d x (A) contains at least two distinct points. 

(7) If A is infinite then the sets d x (A) and Cdx(A) are A-invariant. 

PROOF. Statement (1) follows from the fact that in these circumstances, X and G 
are quasi-isometric (see for instance [ECHLPT, Theorem 3.3.6]) and so G is word hy- 
perbolic as a group quasi-isometric to a proper hyperbolic metric space [GH, Chapter 5, 
Theorem 12]. Statements (4), (5) and (7) are immediate from the definitions, while (2) 
follows immediately from (6). Statement (3) follows from the fact that an infinite cyclic 
subgroup is quasiconvex in G (though this is not used in 2. 1 ) and from [CDP, Chapter 3, 
Theorem 2.2]. Statement (6) is a consequence of Proposition 2.0(9) together with (3) and 
(4). 

We give a sketch of a proof of (6) independent of Proposition 2.0(9), using Proposi- 
tion 2.0(8), which is easier to prove. 

If A has an element of infinite order, the statement is trivial. Suppose now that every 
element in A has finite order. Since there are only finitely many conjugacy classes of 
elements of finite order in G, we can find an integer N such that the order of any element 
of A divides N. 

Fix a basepoint p £ X. Since Ap is infinite and X U dX is compact, the set dx(A) 
contains at least one point q. Assume q = dx(A). Then Aq = q. Fix a geodesic ray 
r = \p, q) in X. Let A = {a,|/ = 1, 2, . . .} be an enumeration of A such that a, ^ <tj when 
I ^ j. Since a t q = q the rays a,r and r are 25-close far away from p. Thus the action of 
a, on r far away from p is 45-closc to a translation by /, . Since the action of G on the set 
of vertices in X (and so in r) is free and A is infinite, we conclude that lim,-^ /, = oo. 
Thus for big enough i we have /, > 1000M5. But for such /' the action of on r far away 
from p is 1 6MS-close to a translation by M t . This contradicts the fact that af = 1 and (6) 
is proved. ■ 

As the set Cdx(A) is ^-equivariant, the quotient space Cdx(A)/A inherits a metric 
from the restriction of d to Cdx(A): put d(Ax,Ay) = \ni{d{a\x,ajy) \ a\,ai G A} for 
x,y £ X. 

Clearly if x,y € A' then \x\{{d(a\x,a2y) \ a\,ai G A} = inf{d(x,ay) \ a € A}. 
Moreover, for any x,y € X inf {d(x, ay) \ a 6 A} = mm{d{x,ay) \ a € A}. Indeed, let 
M = inf {</(*, ay) | a € A}. Then there is a 0 £ A such that d(x,a Q y) < M+ 1. Since 
the closed metric ball B = B(y, 2M + 2) is compact and the action of G on X is properly 
discontinuous, the set {a G A \ d(a 0 y,ay) < 2M+2} is finite. Therefore the set {a e A \ 
d{x,ay) < M+ 1 } is finite and so inf '{d(x, ay) \ a G A} = min{d(x,ay) | a G A}. Thus 
whenever d(Ax,Ay) = 0 we have Ax = Ay. Besides for any z G X, a x G A d(Ax,Ay) = 
mm{d(x,ay) | a G A} < d(x,a\z) + mm{d{ay,a\z) \ a G A} = d(x,a\z) + d(Ay,Az). 
Since a\ £ A was chosen arbitrarily, this implies that d(Ax,Ay) < d{Ax,Az) + d{Az,Ay). 
Thus (Cdx(A)/A, d) is indeed a metric space. 
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We are now in a position to prove an improved, metric version of Proposition 2.4 (the 
necessary condition has become an equivalent condition). 

PROPOSITION 3 .4. Let Gbea word hyperbolic group and A be a subgroup of G. The 
following conditions are equivalent 

(i) A is quasiconvex in G; 

(ii) for any proper 8-hyperbolic space X and for any properly discontinuous cocom- 
pact action of G on X by isometries the quotient space Cd x (A)/A has finite di- 
ameter; 

(Hi) for some proper 8-hyperbolic space X and some cocompact properly discontinu- 
ous action ofGonX by isometries the quotient space Cdx(A)fA has finite diam- 
eter. 

Proposition 3.4 provides an alternative definition of quasiconvexity which highlights 
the connection of the theory of word hyperbolic groups with the theory of classical hy- 
perbolic groups (i.e. discrete groups of isometries of H"). 

Before proving Proposition 3.4, we must do a little preliminary work. 

Recall that if (X, d),(Y, d\) are metric spaces, a map /: X — ► Y is called a K-quasi- 
isometry of X if there is K > 0 such that for each x u x 2 G X (1 /K) d(x u x 2 ) - K < 
di(f(xi),f(x 2 ))<Kd(x u x 2 ) + K. 

The map / is called a K-quasi-isometry of X onto Y if in addition the image f(X) is 
K-cobounded in Y, that is, for each y G Y there is x G X such that d\ (y ,/(*)) < K. In 
this latter case we say that the two spaces are quasi-isometric. 

A /f-quasi-isometry of a connected subset of the real line to Y is called a K-quasigeo- 
desic. In a 6-hyperbolic spaced, there is a constant e, depending on A' and K , such that if 7 
is a /C-quasigeodesic joining two points x, y, and or is a geodesic joining the same points, 
then or is contained in a e -neighbourhood of 7, and 7 is contained in a c -neighbourhood 
of a. For a proof of this essential result ("quasigeodecis are close to geodesies"), and 
more about quasi-isometries, see [Gr, Chapter 7], [GH, Chapter 5], [CDP, Chapter 4], 
etc. 

A proper metric space (X, d) is said to be A-quasigeodesic (A > 1), if for any pair of 
points x, y G X there is a sequence x = x 0 , x\, . . . ,x n = y such that </(jc ( ,Jt/+i) < A for 
i = 0, . . . , n — 1 and |/' —j\ < A d(Xi,Xj) + A for any ij. 

The following lemma will be needed in the proof of 3.4 (it is analogous to Theo- 
rem 3.3.6 of [ECHLPT]). 

LEMMA 3.5. Suppose a group G acts by isometries on the \-quasigeodesic space X 
properly discontinuous ly and cocompactly. Choose a basepointp G X Then the following 
hold: 

(1) The group G is finitely generated. 

(2) If S is a finite generating set of G then the map f : (G, ds) — + (X, d) defined by 
f g*~^ gP is a quasi-isometry of the Cayley graph (Vs(G), ds) onto (X, d). 

PROOF. We use B(x, r) to denote the closed metric ball of radius r centered at x. 
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To see that G is finitely generated we follow the chain argument as in [ECHLPT, 
Theorem 3.3.6]. PutM = diam(*/G). PutS = {g G G \ B(p,M+X)ngB(p,M+X) 4 0}. 

The set S is finite since B(p, M + X) is compact and the action of G is properly discon- 
tinuous. 

We claim that S generates G. Indeed, let g G G. Consider a A-quasigeodesic sequence 
p = jto, x\,...,x n = gp connecting p and gp. Put B = B(p,M). Clearly GB = X. 
Therefore for any i = 1 , . . . , n — 1 there is g, G G such that jc, G gtB. Put go = 1 , gn = g- 
Then (x,+i,g,/>) < </(x,+i,x,) + d{Xj,gip) < M + X and </(x,-+i,g,-+ip) < A/ for / = 0, 
!,...,»- L Thus gr 'jfj+i G fl(p, Af + A) n B{g, ~ 1 g,+ 1/?, M) and so /i, = g^'ft+i e 5 - But 
clearly g = /i 0 /»i • • ■ h n -\ and hence G is generated by 5. 

We will prove statement (2) of Lemma 3.5 for this particular choice of a finite gener- 
ating set S of G. The general statement will then follow immediately from the fact that 
changing a finite generating set of a group induces a quasi-isometry of the word metrics. 

Put N = max{d(p,sp) \ s G S}. By definition the set S is symmetric that is 5 = S~ l . 
Let gi,g 2 G G. We need to estimate d(g\p,g2p) in terms of ds{g\,g 2 ). If s\ • ■ • s k is a 
tfe-geodesic representative for gf'g 2 then puty 0 = g\p, yi = g\S\ • • • s { p for / = 1 , . . . , k. 
Then d{y h v l+ i) = d{p,s i+l p) and so d(gip y g2p) = d(y 0 ,y k ) <Nk = N dsigi,g 2 ). 

Now we need to estimate d s (g\,g2) in terms of d(g\p,g 2 p). Consider a A-quasigeo- 
desic sequence zq = g\p, ...,z„ = g 2 p connecting g\p with g 2 p (instead of a geodesic 
path [gi/>,g#>] as it is done in [ECHLPT, 3.3.6]). Now for any /' = 1, ... ,n — 1 there 
is fi G G such that z, G fiB{p,M). Put yb = guf n = g2 and hi = f~ x fi+\. As before, 
d(fiP,fi+\p) < 2A/ + A and hence hj G S for each i = 0, 1, . . . ,n — 1. Again g]"'g2 = 
hoh | • • • /i„_ i and so </$(gi , g 2 ) < n < A </(gi/>, g 2 p) + X since the sequence zo, . . . , z„ is 
A-quasigeodesic. Thus 

(\/N)d(g iP ,g2p) < djgugi) < Xd(g\p,g2p) + X. 

It remains to note that the set f{G) = Gp is cobounded in G. But we have already seen 
that GB = GB(p, M) = X and so for any x G X there is gp G such that d(x,gp) < M. 
This completes the proof of Lemma 3.5. 

We need to introduce one extra piece of notation before proving Proposition 3.4. If Y 
is a set and d\ , d 2 are real-valued functions on Y x Y we term them equivalent and write 
di - d 2 if for some C > 0 we have {\/C)d x - C <d 2 <Cdi+C. Also if Z is a set, 
Y is a set equipped with a real-valued function d on Y x r and h: Z -> K is a map, we 
term the function (/,onZxZ given by d x {z u zi) = J(/j(z,),/z(z 2 )) the h-pullback oi d 
and denote it /»,(</). 

Thus, for instance, two finite sets of generators for a finitely generated group give rise 
to equivalent distance functions. 

Proof of Proposition 3.4. (i) implies (ii): Let A be a quasiconvex subgroup of G 
and suppose that G acts on A' as in (ii). If A is finite the the statement is obvious since 
d x (A) is empty. Assume now that A is infinite so that it has at least two distinct limit 
points in dX. Pick a basepoint p G X so that p G Cd x (A) and a finite generating set S 
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for G. As in Lemma 3.5, the map/: (G, d s ) — » (X, d) given by f(g) = gp is a Af-quasi- 
isometry for some > 0. Consider a point x G CdxiA); by definition x G (/i,<7i) for 
some/i,<7i € dx(A). This implies that for some points / € [x,/i)and<jr G [x,^i) far from 
x, and for some a, b £ A, the values (/, ap) x , iq, bp) x are big. 

More precisely, for some L > 10005, the points t,q and a, b G A are such that 
('» «^)jc > (<7> bp) x > L. Consideration of the quadrilateral formed by ap, bp, t and 
q, shows that there is a pointy G [ap, bp] with d(x,y) < 68. 

Consider now a ^-geodesic sequence of points go = a, g\, . . . ,g„ = b in G, that 
is d s (gi,gj) = \i Since / is a tf-quasi-isometry, the broken line a = [ap,g\p] U 
\g\Pi g2p] U ■ • U \g„- \p, bp] is a 2KM-quasigeodesic where M = max{J(p, sp),s eS}. 
As quasigeodesicsare close to geodesies, there is a constant e = e(X, K, M) such that a is 
t -close to the geodesic [ap, bp], and so there is k, 0 < k < n such that d(y, g&) < A/+ e. 
Therefore d(x,gkp) < 66 + M + e. Now y4 is e-quasiconvex in (G, d s ) and go = a, 
g\,--,gn = b is a </s-geodesic sequence. Therefore there is a\ G A such that </s(g*, <*\ ) < 
e. This implies d(g k p,a\p) < Ke + K and so d(x, a\p) <66 + M + e+Ke + K = Ki. This 
implies dian^Cd*^)/^) < K\ since x £ dx(A) was chosen arbitrarily. 

(ii) implies (iii): Let (A', d) be the Cayley graph of G with respect to some finite gen- 
erating set. Then X is 6-hyperbolic for some 8 > 1 , geodesic, has compact closed metric 
balls and the action of G on X by left multiplication is properly discontinuous and co- 
compact. 

(iii) implies (i): Assume (X, d) and an action of G on X are as in (iii). Fix a finite 
generating set S for G and word metric d s on G. If A is finite then it is obviously quasi- 
convex in G. Assume now that/i is infinite. By Lemma 3.3, the boundary dx(A) contains 
at least two points and so C = CdxiA) is nonempty. Moreover C is 1006-quasiconvex in 
^Tby Lemma 3.1. Therefore the closure (in X) C is 102^-quasiconvex and /f-invariant. 
It is easy to see that (C,d\ c ) is A-quasigeodesic for A = 100006, as follows. Con- 
sider any two points x, y e C and a geodesic segment [x, y] in X. Choose a sequence 
vo = x, vj,. . . ,v„ = y of points on [x,y] so that ^(v B _i,v„ < 1 and 4(Vj,V#ti) = 1 
for i = 0, ...,» — 2. Then for each /'= 1 ,...,«— 1 there is a point x, G C such that 
</(v,,x,) < 1026. Put xo = x and x„ = v. Then the sequence x = x©, xj, . . . ,x„ = v is a 
required A-quasigeodesic sequence connecting x and v inside C (in fact it is a (2046 + 1 )- 
quasigeodesic). 

Now C is ^-invariant and C/A is compact since C/A is bounded and X has compact 
closed metric balls. Thus Lemma 3.5 implies that A is finitely generated. Pick a finite 
generating set T for A. Take a point p G C and define /: G — > X by /(g) = gp. Notice 
that/(/l) C C since p G C and C is /l-invariant. 

The word metric d T on /I is equivalent to the/-pullback of the restriction of d to C by 
Lemma 3.5. On the other hand the word metric d s on G is equivalent to the /-pullback 
ofrfby [ECHLPT, 3.3.6]. 

Therefore 

d T ~fMc)=M d )U~ d sU- 
Thus </ r ~ dsU and so /I is quasiconvex in G. 
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LEMMA 3.6. Let Xbe a b-hyperbolic space (b > \) and K be a closed subset ofdX 
with at least two elements. Let C = CK and K\ be the limit set of C in dX, that is K\ is 
the union of all limits of sequences of points oj'C converging to infinity. 

ThenK = K x . 

Proof. Clearly K c K x . 

Pick a point p G C and put C, = CU (Ueff[p,a)). Then C { is contained in the 1 OS- 
neighbourhood of C since ideal triangles in X are 1 OS-thin. Indeed,/? G (x,y) for some 
x,y G K. Let a G K be an arbitrary element different from x,y. Then the ideal triangle 
with sides \p,x), [p, a) and (jc, a) is 1 OS-thin (here \p,x) C (x,y)). Thus [p, a) is contained 
in the 1 OS-neighbourhood of \p, x) U (jc, a) and so in the 1 06-neighbourhood of C. 

Suppose b = lim/^ooJt, where x, G C and b G By the previous remark, we 
may assume that each jc, lies on a geodesic r, = [p, a,) for some a, G Since SA' is 
compact, after passing to a subsequence, we may assume that lima, = a. Clearly a G K 
since K is closed. Fix a geodesic ray r = [p,a). We are going to show that a = b, and 
thus that K\ = K. Since lim a, = a, we conclude that lim(r,, r) p = oo. If b ^ a and 
jc, — ► b, there is M > 1,M < oo such that (jr,,r)p < A/ for each /'. Now d(p,Xj) — ► oo. 
Therefore we may assume, after passing to a subsequence, that for each /(jc, , r) p < M and 
d(p,Xi) > 1000MS. But the last two inequalities imply that (r,,r) p is 1 OS-close to (jc,,r) p 
which contradicts lim(r„ r) p = oo. ■ 

Recall (see [Bo2] for details) that a discrete group G of isometries of a ^-dimensional 
hyperbolic space H" without parabolics (that is any element of infinite order fixes exactly 
two points in the boundary 5"-') is termed geometrically finite if Conv(A(G))/G is 
compact, where A(G) is a limit set of G in and Conv(A(<7)) is a (classical) convex 
hull of the union of all geodesies with endpoints in A(G). The following theorem, which is 
due to G. Swarup [Swa], follows immediately from our new definition of quasiconvexity. 

Theorem 3.7 [Swa]. Let Gbea torsion free geometrically finite group of isome- 
tries of H" without parabolics. Then G is word hyperbolic and a subgroup A of G is 
quasiconvex in G if and only if A is geometrically finite. 

PROOF. Take X to be the (classical) convex hull of the limit set of G. As a convex 
subset of H", X is 1 0-hyperbolic with the metric induced from H". The group G is geomet- 
rically finite and without parabolics. Thus the action of G on X '\s properly discontinuous 
and X/ G is compact. 

Therefore by Theorem 3.3.6 of [ECHLPT] G is finitely generated and has a Cayley 
graph which is quasi-isometric to a 1 0-hyperbolic metric space X, so that G is word 
hyperbolic. The stated properties of quasiconvex subgroups now follow directly from 
Proposition 3.4 and the definition of geometrical finiteness. 

Indeed, if A < G is geometrically finite then Conv(A(A))/A is compact and so 
has bounded diameter M < oo. On the other hand A(A) = d x {A) = K and CK C 
Conv(A(,4)) is A -invariant. Thusdiam(CK//l) < M and/1 is quasiconvex in G by Propo- 
sition 3.4. 
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Suppose now that A < G is quasiconvex in G and A(A) = dx(A) = K as above. 
Then dmm(CK/A) = N < oo. It is easy to see that CK C Conv(A(/l)) and Conv(A(/l)) 
is contained in a 1 OOOw-neighborhood of CK (where n = dimH"). This implies that 
Conv(A(/i)) I A has bounded diameter and so is compact. Thus A is geometrically finite. 

We now obtain the analogue of Lemma 2.2 (ii). 

LEMMA 3.8. Suppose G is a word hyperbolic group acting by isometrics on a proper 
8-hyperbolic space X so that the action is properly discontinuous and cocompact. Let A 
bean infinite subgroup of G, K = dx(A) and H = Stab c (AT) = {g G G \ gK = K}. Then 

(1) For any subgroup E of H, containing A, dx(E) = dx(A). In particular d x {H) = 
dx(A). 

(2) If A is normal in a subgroup EofG then 8x(E) = dx(A). 

PROOF. ( 1 ) Let C = CK. Notice that K has at least two points so that C is nonempty. 
Pick a point peC.lt is clear that K = dx(A) c dx{E). Since K is E invariant, C = CK 
is also £-invariant. Thus Ep C C and therefore dx(E) is contained in the limit set of C 
which by Lemma 3.6 is equal to K. Therefore dx(A) = K = dx{E). 

(2) If A is normal in £, for any g G E we have gAp = Agp. The limit set of Ap is 
equal to the limit set of Agp and is equal to K. Thus gK = K and so A <E< Stob G (K). 
By Part (1) this implies d x (A) = dx(E) = K. 

Commensurability in hyperbolic groups. It is possible to obtain the following crite- 
rion of commensurability for quasiconvex subgroups of hyperbolic groups; this is the 
promised strengthened form of 2.4. 

LEMMA 3.9. Let G be a word hyperbolic group and let E, H be subgroups of G 
with H < E. Let (X, d) be a proper 6-hyperbolic metric space (b > X) equipped with 
properly discontinuous isometric cocompact action of G. Suppose that H is infinite and 
quasiconvex in G and let K = dx{H). Then the following conditions are equivalent. 

(a) H has finite index in E; 

(b) the set C = CK is E-invariant; 

(c) E is quasiconvex in G and dx(E) = dx{H); 

(d) E is contained in Stabc(AT) where Stab c (/Q = {g G G \ gK = K). 

PROOF. Notice that if p G X, h G H then Hp = hHp and so K = hK. Thus H < 
Stab G (AT). Now the implications (a) (b), (a) (c), (a) =» (d), (d) (b) and (c) => (b) 
are clear. 

Suppose now that (b) is satisfied. Then the closure (in X) C is £-invariant and quasi- 
convex in X. Thus diam(C/£) < diam(C///) < oo and therefore CjE is compact. This 
implies by Lemma 3.5 that E is finitely generated. Moreover, if Y is a finite generating 
set for G, Z is a finite generating set for E, p G CK and/: G — » X is given by f(g) = gp, 
then again by Lemma 3.5: 

d Y ~Md), d z ~ (f>(d\c)\E) ^d so d 2 ~ (/.(<%)) I* = fc(«0]e ~ (d Y )\ E . 
Thus E is quasiconvex in G. 
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Observe that K is a closed subset of dX as it is a limit set of a subgroup and so by 
Lemma 3.6 the limit set of C is K. We claim that d x (E) = K. Clearly K C d^E) since 
H < E. On the other hand C is ^-invariant, p £ C and Ep C C. The limit set of E which 
is, by definition, the limit set of Ep is contained in the limit set of the larger set C, and 
this limit is K. Thus d x (E) = K. 

We have shown that (b) implies (c). We will now show that (b) implies (a). 

Indeed, suppose Y and Z are as above and 5 is a finite generating set of H. Let e be an 
arbitrary element of E. Since diam(C///) < M < oo and f(e) = ep G C, there is a point 
hp £ C for some h G H such that d(ep,hp) < 2M. Since / is a A-quasi-isometry from 
(G, d Y ) onto (Af, d), we conclude that </ y (e, h) < 2MX + A = M\. Thus for any e G £ 
there is an element g e € G with </(&,, 1) < A/] such that G //. Therefore // has finite 
index in E. This completes the proof of Lemma 3.9. 

COROLLARY 3.10. Let G be a word hyperbolic acting by isometries properly dis- 
continuously and cocompactly on a proper hyperbolic space X. Let H be an infinite qua- 
siconvex subgroup of G and K = dx{H). Then Stab c (/Q = VN G {H). 

PROOF. It is clear that H < VN G (H) < Stab G (AT). On the other hand H has finite 
index in Stab c (/Q by Lemma 3.9(d). Thus for any g G Stab c (AT) we have \H:gHg~ l D 
H\ < oo and \gHg-* : gHg~ x n H\ < oo. Therefore g G VN G (H) and so VN G {H) = 
Stab c (/0- 

Theorem 1 now follows immediately from this proposition, as in Section 2. 

Proof of Theorem 1 . Let A' be the Cayley graph of G with respect to some finite 
generating set S. Put K = d\{A). 

(1) It is obvious that dx{A) = d x {B) - dx(A D B). Denote this set by K. Thus A, B < 
Stab c (/0 and so E = sgp(A U B) < Stab G (K). Thus by Lemma 3.9(d) AC\B has finite 
index in E. 

(2) Suppose there are infinitely many subgroups of G containing A as a subgroup 
of finite index. Let E be the subgroup of G generated by their union. Then A has infinite 
index in E. On the other hand any subgroup of G having A as a subgroup of finite index has 
K as its limit set and so leaves K invariant. Thus K is ^-invariant, that is E < Stabc(/Q. 
Therefore A has finite index in E by Lemma 3.9(d). This contradicts our assumptions. 

(3) It is clear that /I < VN G (A). Suppose now thatg G VN G {A). Put U = gAg~ l D H. 
Then V has finite index in A and in gAg~ 1 . Since U is of finite index in A and in gAg~ 1 , 
K = dx{U) = d x (A) = dxigAg~ x ). However dxigAg~ l ) is equal to the limit set of gA 
that is to gA:. Thus gK = K and g G Stab c (AT). Since g G VN G (A) was chosen arbitrarily, 
we conclude that A < VN G (A) < Stab c (/Q. Thus by Lemma 3.9 A has finite index in 
VN G (A). 

Proof of Theorem 3. Let AT be the Cayley graph of G with respect to some finite 
generating set S. Put K = d x (A). The "if implication is clear. So suppose that A contains 
an infinite subgroup C which is normal in B. Then by Lemma 3.8 d x (Q = d x (B). Since 
C < A < B, we conclude that K = d x (A) = d x (B). Thus B < Stab c (/0 and by 
Lemma 3.9(d) A has finite index in B. 
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4. Some additional remarks about the elementary boundary. Let S be a finite 
generating set for the word hyperbolic group G and (X, d$) be the Cayley graph of G 
with respect to S. Fix a number 8 > 1 such that (A', ds) is ^-hyperbolic. The elementary 
boundary dG (in the sense of Section 2) is a subset of dX (in the sense of Section 3) and 
moreover, a dense subset (see [Gr]). The map from the elementary boundary into the 
Gromov boundary is given by [hg + ] ^— » lim,,*.^ hg"h~ y = linv-<» hg". 

It is also not hard to see, using the compactness argument, that if g G 9fc(Y) for a 
subset Y of 8G then there is a pair of points a(g),a(h) € Y such that g lies on a certain 
geodesic (a(g),a(h)) (this is a bi-infinite geodesic the sense of Section 3). Moreover, if 
a(g), a(h) G Y are two distinct points, then there is a geodesic (a(g), a(h)) such that any 
point (vertex) of (a(g),a(hj) belongs to !Hg{Y). 

This immediately implies 

LEMMA 4. 1 . Let G,X,8 be as above, and let Y be a subset of dG with at least two 
elements. 

Then Hj(Y) is \Q08-quasiconvex in X. 

PROOF. The proof is exactly the same as that of Lemma 3.2 if one takes into account 
the remarks above. 

The previous observation combined with Corollary 2.4 allows one to obtain a criterion 
of quasiconvexity similar to Proposition 3.4 in terms of elementary boundary, making the 
condition of 2.4 necessary and sufficient. 

PROPOSITION 4.2. Let Gbea word hyperbolic group, S a finite generating set and 
X be the Cayley graph of G with respect to S. Let A be an infinite subgroup of G. 
The following conditions are equivalent: 

(i) A is quasiconvex in G; 

(ii) the quotient !H0 G A)/A is finite. 

PROOF. The implication (i) (ii) was established in Corollary 2.4. Suppose now 
that (ii) is satisfied. As A is infinite, BA contains at least two points and soC=9{ (B G A) 
is nonempty and even infinite. Now C is 306-quasiconvex in X by Lemma 4. 1 and so 
it is A-quasigeodesic for A = 30006 (the argument is exactly the same as in the proof 
of Proposition 3.4). Now C is discrete (as C C G) and has compact metric balls. Fix a 
basepointp € C. We know that C/Ais finite and therefore compact. Thus by Lemma 3.5, 
A is finitely generated and the orbit map a >— > ap, a € A is a quasi-isometry of A onto C. 
The rest of the argument goes exactly as in Proposition 3.4. 
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THE RESIDUAL SPECTRUM OF G 2 



HENRY H. KIM 



Abstract. We completely determine the residual spectrum of the split exceptional 
group of type G2, thus completing the work of Langlands and Moeglin-Waldspurger on 
the part of the residual spectrum attached to the trivial character of the maximal torus. 
We also give the Arthur parameters for the residual spectrum coming from Borel sub- 
groups. The interpretation in terms of Arthur parameters explains the "bizarre" multi- 
pncity condition in Moegiin-waiuspurgcr s work, it is reiaieu to tne tact mat tne com- 
ponent group of the Arthur parameter is non-abelian. 



1. Introduction. Let F be a number field and (A) its ring of adeles. Let G be a 
reductive group. A central problem in the theory of automorphic forms is to decompose 
the right regular representation of G(A) acting on the Hilbert space L 2 (G(F)\G(\j) . It 
has the continuous spectrum and the discrete spectrum: 

L 2 (G(F)\GW) = L 2 dis (G(F)\GW) 

We are mainly interested in the discrete spectrum. Langlands [L4] described, using his 
theory of Eisenstein series, an orthogonal decomposition: 

where (M, n) is a Levi subgroup with a cuspidal automorphic representation n taken 
modulo conjugacy. (Here we normalize 7r so that the action of the maximal split torus in 
the center of G at the archimedean places is trivial.) Lj is (C/(F)\<j(A)) j s a space of 
residues of Eisenstein series associated to (A/, it). Here we note that the subspace 

e4,(G<f)\C<A)) (Ci) , 

is the space of cuspidal representations Z4 sp (G(F)\G(A)) . Its orthogonal complement in 
L 2 dis (G(F)\G(f^)) is called the residual spectrum and we denote it by L^.(G(F)\G(A)). 
Therefore we have an orthogonal decomposition 

4 S (G(F)\G(A) = L 2 cusp (G(F)\GW) Qll,(G(F)\G{\)). 

Arthur described a conjectural decomposition of this space as follows: 

4(^)) = 0£ 2 (C(F)\C(A))^ 

Partially supported by NSA grant MDA904-95- 1-1078 
Received by the editors January 25, 1 996. 
AMS subject classification: Primary: 1 1F70, 22E55. 
© Canadian Mathematical Society 1996. 
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where tp runs, modulo conjugacy, through the set of moronism: 

xp:L F x SL(2,C) ■— * G*, 

where Lf is the conjectural tannakian group, G* is the Langlands' L-group and ip satisfies 
certain conditions; in particular, t/; restricted to SL(2, C) is algebraic, ip restricted to L F 
parametrize a cuspidal tempered representation of a Levi subgroup and the image of \p is 
not included in proper Levi subgroups. The space L 2 (G{F)\G(h))^ is defined by local 
data (See Section 3.7). 

The purpose of this paper is to determine explicitly L]Jfi(F)\G{\)) for G the split 
exceptional group of type G 2 . There are 3 Levi subgroups modulo conjugacy. Let M\ be 
the Levi of the maximal parabolic subgroup P\ attached to the long simple root and A/ 2 
be the Levi of the maximal parabolic subgroup P 2 attached to the short simple root. Let 
M 0 = T be the maximal split torus. Let 

lL(g{F)\gw) Mi = 04(WW) (Mf) . 

Theorems 3.6.1, Theorem 4.2 and Theorem 5.1 describe the decomposition of 
£d is ((7(F)\G(A)) w for /' = 0, 1,2, respectively. Due to the lack of information on the 
poles of the adjoint cube /--function of GL2 (symmetric cube /.-function of GL2 twisted 
by a character) and insufficient bound for Fourier coefficients of cuspidal representations 
of GL 2 , our results on Lj is (G(F)\G(A)) w , / = 1,2, are incomplete. Also we only con- 
sider the ^-finite, /^-invariant subspace of Lj^(G(F)\G(K)) . We also give the Arthur 

parameter for automorphic representations in Z,j is (G(F)\G(A)) _ and verify Arthur's con- 
jecture, reformulated by Moeglin [M3]. 

In order to obtain the decomposition for L^ S (G(F)\G(A)) , we use the inner product 
formula (3.1)ofthe pseudo-Eisenstein series as in [M-Wl]. ForZ,j js (G(F)\G(A)) w , / = 
1, 2, we use the method in [Ki], where we calculated the residual spectrum of Sp 4 . We 
use the notation of [M-Wl]. Let us explain in detail in each of the cases. 

The most interesting one is the analysis of lJ is (G(/ r )\G(A)) r It was Langlands [L4, 
Appendix 3] who first calculated its AT-fixed subspace. It is dimension 2. One is spherical 
and the other is a very interesting residual automorphic representation. Its Archimedean 
component is infinite dimensional, of class one and is not tempered. Moeglin and Wald- 
spurger [M-Wl, Appendix III] calculated ^-finite, -invariant subspace V of 
Lli S (G(F)\G(&)) {T 1} » where 1 is the trivial character of T, whose cuspidal exponents 
are short roots. They found surprising results that only those which satisfy a certain con- 
dition appear in Z,J is (G(F)\G(A)) 7 .. Let us explain in detail. LetJ v = {it\ v , 7r 2v } be a set 
of irreducible representations of G v , where n\ v is spherical. For S a finite set of finite 
places, set 7^ = it\ v <g> (g)^^ 7^. Then 

V= 0 7T 5 . 
S,card(StfI 

The condition card(S) ^ 1 is quite surprising (In Sp 4 case [Ki], we have the condition 
"card(iS') even"). We can interpret their results in terms of Arthur parameters. In fact, the 
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condition comes from the Springer correspondence (see Section 3.7 for more details) and 
is related to the fact that A(u) is non-abelian, i.e., S 3 , the symmetric group on 3 letters. 
Recall that the Springer correspondence is an injective map from the set of irreducible 
characters of W, the Weyl group of G, into the set of pairs (O, r/), where O is a unipotent 
orbit and ?/ is an irreducible character of A{u) = C(«)/C(«)°, u € O and C(u) is the 
central i/er of u. Let Springer(0) be the set of characters of A{u) which are in the image 
of Springer correspondence. Then J v is associated with Springer((7 2 (ai)), where G2(a\) 
is the sub-regular unipotent orbit of Gi ([Ca, p.40 1 ]). We note that Moeglin [M 1 ] proved 
that the residual spectrum attached to the trivial character of the torus is parametrized by 
distinguished unipotent orbits O and Springer(0) and we can expect that the same thing 
would happen for all split groups. 

Among non-trivial characters of T(\)/T(F), modulo conjugacy, the following char- 
acters of the torus contribute to the residual spectrum (see Section 3.6). Under the iden- 
tification, M\ ~ GL 2 , where M x is the Levi subgroup of P u X = x0*>")> V and v are 
grossencharacters of F. 

(1) ^ = ^=1,^1 

(2) p?-l,njf 

For Case (2), there is only one residual spectrum, which is the global Langlands' 
quotient of Ind^ exp(/?4, ///>,()) ® (I n dfl 0 L2 X)- Case (1) is more interesting. In this case 
the Eisenstein series has a pole at /3 2 , the sum of two simple positive roots. If /i v is 
not trivial, then the character Xv ® exp(/3 2 ,//g( )) is regular and we can apply Rodier's 
result ([R]) to analyze the image of the intertwining operator R(pi, /? 2 , pi\\ In particular, 
it is irreducible. If /i v is trivial, then the image of R(p2,(3i, P2X) is the same as the one 
Moeglin and Waldspurger found [M-Wl, Appendix III]. It is the sum of two irreducible 
representations. Let J(\v) = {*iv,tt2v} is the set of irreducible components. We put 
* 2v = 0 » f Xv ¥ 1 ■ For s a finite set of finite places, set 7T 5 = ® v ^ s ir Xv 0 ®ves *iv Then 
the residual spectrum attached to the character ( 1 ) is given by 

s 

There is no condition on S. 

For Ll is (G(F)\G(&)) , the Levi factor is M\ = GL 2 . We have to look at Eisen- 
stein series associated to cuspidal representations of GL2. The L-functions in the con- 
stant terms of Eisenstein series are the adjoint cube L-function and Hecke L-ftinction. 
We analyze the poles and the irreducibility of the images of local intertwining operators 
as in [Ki]. However, the pole of the adjoint cube Z,-function of GL 2 is not known. We 
assume the location of poles. Also we need to assume certain estimates of Fourier co- 
efficients of cuspidal representations of GL 2 . More precisely, if tt v = x(p| | r ,/i| |~ r ) is 
a complementary series representation of GL 2 , we assume that r < i. Right now the 
best known result is that r < } due to Shahidi [S3]. Assuming these facts, we obtain 
a decomposition of Z4(G(F)\G(A)) W , parametrized by cuspidal representations tt of 
GL 2 with trivial central characters and L(i, tt, r?) ^ 0 and by monomial representations 
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of GL 2 . Recall that a cuspidal representation a of GL2 is called monomial if a ~ a ® ;/ 
for a quadratic grossencharacter 7/ of F. 

For L^ js (G(F)\C(A)) , the Levi factor is Mi = GL 2 . The L-functions in the con- 
stant term of the Eisenstein series are just Jacquet-Langlands' L-function, its twist by 
a character, and a Hecke /--function. In this case, assuming the fact on the estimates 
of Fourier coefficients of cuspidal representations of GL 2 , we obtain a decomposition 
of Lj is (G(F)\G(A)) parametrized by cuspidal representations 7r of GL 2 with trivial 
central characters and L(\,-k) ^ 0. 

F. Shahidi brought to our attention the paper by Li and Schwermer [Li-Sc] who studied 
the poles of Eisenstein series attached to the maximal parabolic subgroups over Q. After 
this paper was accepted, the author received a preprint, "The residual spectrum of the 
group of type G 2 ," by S. Zampera. She obtained similar results. But her result does not 
have the interpretation in terms of Arthur parameters. 

Acknowledgements. The author would like to thank Prof. F. Shahidi and Prof. 
C. Moeglin for engaging in many helpful conversations. Thanks are due to the referee 
for many helpful comments. 

2. Some facts about G 2 ; roots and parabolic subgroups. Let G be a split group 
of type G 2 . Fix a Cartan subgroup T in G and let B = TV be a Borel subgroup of G. Let 
Koo be the standard maximal compact subgroup in G(/V>o) and K v = G(O v ) for finite v. 
The products = /:«> x Y[K V is a maximal compact subgroup in G(A). 

We follow Moeglin and Waldspurger [M-Wl, Appendix III]. Let 0, be a long simple 
root and 0 6 a short one. Let 

02 = 0, +06, ft = 2/3, + 306, 04=01+206, 05 = 0, +30 6 

then {0|, ... ,0 6 } is a set of positive roots. 

Let fff be a corresponding coroot of 0, for i ; = 1 , . . . , 6. Then 

0 2 v = 30y+0 6 v , 0^ = 20^0-, 0 4 v = 30y+ 2 0 6 v , 0^ = 0r + 0 6 v . 

Let P\ the maximal parabolic subgroup generated by 0, (long root) and Pi be the 
maximal parabolic subgroup generated by 06 (short root). Then we have Levi decompo- 
sitions (Shahidi [S4]): 

/>, = M X N U Pi = M 2 N 2 , M\ ~ GL 2 , M 2 ~ GL 2 . 

Under the identification M\ ~ GL 2 , 

(2.1) # (/) " dia 8('' r ')' #(') - diagr',/ 2 ), 0 2 v (/) = diagf/V) 
0 3 / (O = diag(/, 1), 0i(t) = diag(/,/), 0^(/) = diag(l,/). 
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Under the identification M 2 ~ GL2, 

(2.2) = diag(1 '°' = ** r 'X #<') = diagC/,/ 2 ) 

#(/) = diag(/, /), = diagCr 2 , /), fift = diag(/, 1 ). 

Let X(T) (resp. ^(7")) be the character (resp. cocharacter) group of T. Since G is simply 
connected, 

X(T) = Zft + Zft, X*(T) = Zft + Z0l 

Let 

a*=X(T)®R t a* c =X(T)®C, a = X*(T) ® R = Hom(X(T), R) 

a c =X*(T)®C. 

Then {ft,ft} and are the pair of dual bases for a* and a, respectively. 

The positive Weyl Chamber in a* is 

C* = {A € a* I (A,ar v ) > 0 for all a positive roots } 

The half sum of positive roots is ps = ft + ft. 

We list the elements of the Weyl group together with their actions on the positive 
roots. 



w 


decomposition 


?1 


02 


01 


ft 


ft 


ft 


1 




ft 


02 


03 


ft 


ft 


06 


Pi 


Pi 


-ft 


06 


05 


ft 


ft 


02 


P2 


P1P6P1 


"A 


-02 


"ft 


ft 


05 


04 


Pi 


P\PbP\Pbp\ 


"ft 


-04 


-03 


"ft 


-01 


06 


PA 


P6P\P6PIP6 


ft 


-06 


-05 


"ft 


-ft 


-02 


PS 


P6PIP6 


ft 


02 


ft 


"ft 


"ft 


-04 


P6 


P6 


ft 


04 


ft 


ft 


0. 


"ft 




P6P1 


-A 


-06 


01 


ft 


ft 


ft 


P6P1P6P1 


-ft 


-04 


-ft 


"ft 


ft 


ft 


a(TT) 


P6P\P6P\P6P\ 


-ft 


-02 


"ft 


-ft 


"ft 


-06 




PlP6plP6 


ft 


06 


"0. 


-ft 


"ft 


-ft 




PIP6 


01 


04 


ft 


ft 


-01 


-ft 



3. Decomposition of Ll is [G(F)\G( A)) r . We fix an additive character tp = (S> v tp v of 
A /F and let £(z, /1) be the Hecke L-function with the ordinary T-factor so that it satisfies 
the functional equation £(z,/i) = e(z,/i)£(l — z,/i _l ), where e(z,ji) = flv e(z, Pv, Vv) is 
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the usual t-factor (see, for example, [Go, pi 58]). If /i is the trivial character /io, then we 
write simply £(z) for £(z, /io). We have the Laurent expansion of £(z) at z = I : 

c(F) 

3.1. Definition of Eisenstein Series. For a unitary character x of T(\)/T(F) and for 
each A G a* c , let /(A, x) = Ind£ x ® exp(A, )) be the induced representation, where 
H B is the homomorphism H B : ^ a defined by 

exp( X ,// fl (0) = niXv(/v)|v. 

V 

We form the Eisenstein series: 

E(g,f,A)= £ Alg). 

ieB(F)\G(F) 

where / G /(A, x). The Eisenstein series converges absolutely for Re A G C*" + pj» and 
extends to a meromorphic function of A. It is an automorphic form and the constant term 
of E(g,f, A) along B is given by 

*fo/,A) = J u(F)W) E(ug,f,A)du = E%A lX m 

where W is the Weyl group and for sufficiently regular A, 

M(w,A,xV(g) = j Uw{K /i"-'u2)d»> 

where U w = U H wt/w -1 , C/ is the unipotent radical opposed to U. Then M(w,A, x) 
defines a linear map from /(A, x) to I(wA, w\) and satisfies the functional equation of 
the form 

A/(w, w 2 , A, x) = Af(wi , w 2 A, w 2 x)M(w2, A, x). 
The Eisenstein series satisfies the functional equation 

E(g, M(w, A, xV, wA) = E(g,f, A). 

Let 5 be a finite set of places of F, including all the archimedean places such that for 
every v ^ 5, Xv and i/> v are unramified and iff = ® f v , for v £ 5,/ v is the unique K v -f\xed 
function normalized by f v (e v ) = 1 . We have 

A/(w, A, x) = (8) M(w t A, Xv). 

V 

Then by applying Gindikin-Karpelevic method (Langlands [L4]), we can see that for 
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where L(s, t] v ) is the local Hecke Z,-function attached to a character // v of F* and i G C 
and f v is the Affixed function in the space of 7(wA, w\) satisfying/ v (e v ) = 1 . For any v, 
let 

aX),wa<0 

U( A, a v ) + 1 , X v o a v )e«A, « v ), Xv o a v , i/>v) ' 
We normalize the intertwining operator A/(w, A, x v ) for all v by 

M(w, A, Xv) = r v (w)R(w, A, Xv). 

Let R(w, A, x) = ®v R(w, A, Xv) and R(w, A, x) satisfies the functional equation 

R(wi w 2 , A, x) = R(w\ , w 2 A, H>2X)fl(w 2 , A, x). 

We know, by Winarsky [Wi] for p-adic cases and Shahidi [S2, pi 10] for real and complex 
cases, that 

A/(w,A,Xv) II £«A,« V },Xv°aV 

a>0,wa<0 

is holomorphic for any v. So for any v, R(w,A,\ v ) is holomorphic for A with 
Re((A, or v )) > — 1, for all positive a with wot < 0. 

We note that a character x of T(F)\T(fa) defines a cuspidal representation of T. For 
any w G W, wTw~ x = Tand so (T, w\) is conjugate to (T, \). 

Let /(x) be the set of entire functions / of Paley-Wiener type such that /(A) G /(A, x) 
for each A. Let 

where A 0 G Pb + C* ■ Then we have 

LEMMA ([L4]). L 2 (G(F)\G(K)) {T x) spanned by 9/ for all f G I(w\) as w\ runs 
through all distinct conjugates of \. 

4s(<W\G(A)) (7 . x) is the discrete part of Z, 2 (G(F)\G(A)) (r ^ It is the set of iterated 
residues of £(g,/(A), A) of order 2. 

In order to decompose L 2 dis (G(F)\G(fc)) (J . , we use the inner product formula of two 
pseudo-Eisenstein series: Let x and x' be conjugate characters and / G I(x),f G 7( X ')- 
Then 

= 75^2 L A ^ £ (Mw,AWA),/<-wA))JA, 

where W(x,x') = {w G W | w X = x'}- Let {rf X I d G D} be the set of distinct 
conjugates of x- 

In order to deal with the distinct conjugates of x simultaneously, we consider, for 
/ 6 /(X), 

W;A)=£ £ (Mw,A,x/(A),^(-wA)), 
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where f' d e I(d\). Since W = \J JeD W(\, for simplicity, we write it as 

(3. 1) A<fS;A) = Z (M(w,A, X y(A),/(-wA)). 

We also have the adjoint formula for the intertwining operators 

(3 2) (^ A 'Xy(A)/(-wA)) = (f(A),M( W -\-w\, WX )f(-»ty 
(/?(w,A,xV(AX/(-H'A)) = (jr(A),/?(w- , ,-wA,w X y(-wA)). 

We use this adjoint formula and calculate the residue of A(f,f; A) to obtain the resid- 
ual spectrum Ll s (G(F)\G(\)) (Tx) . 

Let Si = {A € a£ : (A, ffj) = 1} and we introduce a coordinate on S t as follows: 
A = zm + f , where u x = ft, "2 = ft, «3 = ft, «4 = ft, «5 = ft, "6 = ft. We note 
that( M/ ,/3 v ) = 0. 

In order to get discrete spectrum, we have to deform the contour Re A = Ao to 
Re A = 0. Since the poles of the functions M(w,A,\) all lie on 5, which is defined 
by real equations, we can represent the process of deforming the contour and the singu- 
lar hyperplanes 5, as dashed lines by the following diagram in the real plane as in [L4, 
Appendix 3]. 

The integral at Re A = 0, 



gives the continuous spectrum of dimension 2. As can be seen in the diagram, if we 
move the contour along the dotted line indicated we may pick up residues at the points 

A/, i = 1 , . . . , 6: 

where A G 5,. Then we deform the contours Re A = A, to Re A = f , i.e., the origin of 
S h The integrals at Re A = §, 



1 

2ixi 



give the continuous spectrum of dimension 1 . The square integrable residues which arise 
during the deformation span the discrete spectrum. 
As we see in the diagram, we have to consider 

Resft Res S] A(f,f ;A) 

Res, 32 Res Sl A{f t f;A) 
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Res. ft Res ft A(f,f\ A) 

Res^Ress, A(f,f; A) 

Res Pfl Res Si /l(/-,/ f ;A) 

Res^Res^ /*(/",/; A) 

Res, ft Res^/l(/",/';A). 
For x a character of T(^/T(F), let x« = X ° 
Then 

X2 = XlX6,X3 = XlX6,X4 = XlX6.X5 = XlX6- 

Set 

Xf(w, A, X ) = ^ Res Sl M(w t A, X ). 
Let Wi = {w<EW\ wpi < 0} for/ = 1,...,6. Then 

Res Si A(f,f, A) = £ (Ufa A, X y(A),/(-wA) 



3.2. Calculation of Res*, A{f,f; A). M(w, A, x) has a pole at 5, if x i 
On Si, 



1254 



HENRY H. KIM 



<A,#)=z+^<A,/3 3 v )=z+i 
<A,/3 4 V > = 2z, <A,#> -z-i (A,/3 6 v ) -*-|. 

Lemma 3.2.1. 

M'(pi,A,x) = /?(pi,A,x) 

, A v- ^ + i,X6)/?(P2,A,x) 
«^ A ^)-^ (z+ | )X6)e(z+ i >X6)e(z+ | X6) 

, £(2z,xgK(z-+,X6) *(P3,A,X) 

(P3 ' ,X) £(z + |,X6X(22+ 1,XIM^+ |,X6Mr+ ^X6)e(22,xi)e(z - \, X <>) 

= gz+i,X6)£(2z,xI) i?(a(|),A,x) 

4(2 + §, X6X(2z + 1 , x\)e{z + |x 6 )e(z + 5, XeW* + J, X6)e(2z, x 2 6 ) 

v 7 C(2+|,X6K(2z+ 1,X6> 

jg(o(^A,x) 

' e(z + |, X6k(z + }, X6k(2z, x?Mz - i, X6)e(2 - §, X6 ) 

PROPOSITION 3.2.2. #x « wo/ /r/v/a/, /Ae/i Res s , A) was a pole atz=\, i.e. 

A = fa w/iew X £ = I, X6 ^ 1 residue is given by 

Resfc Resj, A(f,f ; A) = c,(fl0 2 ),*(P2,02,02X)) 

^(p 5 ,"ft ", x V0&) + j),0 4 , P3x)AA0 + c 3 /?(p 6 ,/? 4 , y xy^4)j , 

where c u c 2 ,a are constants and R(p 5 , "fa ". X) = ®v*(P5. "fa'\ Xv )and R(p 5 , "fa", 
Xv) ^ va/we a/ /? 2 <?/ restriction /?(p 5 , A, Xv)|s, • /* & a« isomorphism from I(j3 2 , X ) 
toI(0 2 ,Ps X ). 

PROOF. The local intertwining operators are holomorphic at z = \ except 
/?(<t(7t), A, Xv) when Xv - 1. However, Moeglin-Waldspurger [M-Wl, Appendix III] 
showed that the restriction R(o(n), A, Xv)|s, is holomorphic at z = \: Let Xv = I. By 
the cocycle relation, R(o(ir), A, Xv) = R(Ps, PiK X v)R(P2, A, Xv)- The pole is contained 
in /?(p 5 ,p 2 A, X v). However, the restriction /?(ps,p2A,Xv)U, of fl(ps,P2A,Xv) onto S\ is 
holomorphic at fa. If Xv is not trivial, /?(p 5 , p 2 A, Xv) is holomorphic at fa. 
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Therefore, all the poles are contained in the normalizing factors. So 
Res* Res, A(f,f; A) = 



^(2K(3, X 6M2,X6) 

In order to use the adjoint formula (3.2), we note that #3 = p 2 o{\ ) and <r(y) _l = 
<r(y ) = P2P6- We also note that <r(7r) = pip$ = p$pi and so we have R{a(ir), A, Xv) = 
R(pi, PsA, PsXvMps, A, Xv). By the same reasoning as above, if Xv - 1, *(P5, A, Xv)|s, 
is holomorphic at ft and we denote its value by /?(p 5 , "ft", Xv). If Xv is not trivial, then 
R(ps, A, Xv) is holomorphic at ft and in this case, R(p 5 , ft, X v) = R(Ps, "ft", Xv). 

Proposition 3.2.3. If\ is trivial, then Res Sl A(f,f\ A) has a pole atz = \, i.e. at 
0 2 andz = \, i.e., at ft. 

(1) Res* Res Sl A{f,f ; A) was calculated by Moeglin-Waldspurger. 

(2) Res* Res Sl A(fJ*;A) = 0. 

Proof. (1) See [M-Wl, Appendix IV]. 

(2) At z = 5, A/'(p 3 , A, x) and M l (a(n), A, x) have simple poles. So the residue is 
given by 

Res* Res* A) = ^^^((j^,A,x)-^(<KitXA.x))/WjX/<»j)). 

But/?(<T(7r),ft,x) = K(P3P6,ft,x) = /?(p 3 ,ft,x)^(P6,ft,x)and/?(p 6 ,ft,x) = idby 
the following Lemma. So Res* Res$, A(f,f ; A) = 0. 

Lemma 3.2.4. Suppose a character \v satisfies p^Xv = Xv- Thenfor any real number 
t, the normalized intertwining operator R(p(,, /ft , Xv) " a self-intertwining operator of 
/(/ft, Xv)- it acts like the identity. 

Proof. Since p6ft = ft, it is a self-intertwining operator of /(/ft,Xv)- Since 
(ft,/?$) = 0, /?(P6,'ft,Xv) is actually an intertwining operator for the Levi subgroup 
M 2 ~ GL2, where P 2 = M 2 N 2 is the maximal parabolic subgroup attached to ft. Under 
the identification M 2 ~ GL2, R(p6, 'ft, Xv) is an intertwining operator of Indg p| |' x 
p| |'. Since Ind£ o L2 p| |' x p\ \' is irreducible, /?(p 6 , 'ft, Xv) acts like a scalar. But it acts 
like the identity on the K-fixed vector. Therefore it is the identity. 

3.3. Calculation o/Res* A(f,f ; A). M(w, A, x) has a pole at S 2 if \2 = 1 , i.e., x 1 X6 = 
1. 

OnS 2 , 

(A,^> = -z + i,(A,^)=z + I 
(A,ft v ) = 3z+i <A,#) = 2z,<A,ft v ) = 3z- I. 
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Lemma 3.3.1. 

lAt a i- fl-z+s.XiK(z+;.xr') R (P2>^X) 

( " 2, ,x) «-*+|.xi)efr+|.xr , M-*+|.xi)^+i.xr l ) 
M 2 r A i.xr'KQz* j,xr 3 )^xr 2 ) 

tp3 ' ,x) ^(-^^xO^+f.xr'KOz+^xr^+uxr 2 ) 

* £ (-z+ i, x ,Mr+ i,xr*M3^+ |,xr 3 M^xr 2 ) 
(p " ,x) " f,xr')C(2z + i,xr 2 )^(3z + §,xr 3 ) 

* £ (z + 1, xr'M3z + £. xr 3 )^(2^ xr 2 )^ - i. xr 3 ) 
i^U?Uxl C i" Z tl Xl) i .*f»(?Ux1 

I V3/' '*J a _ z+ |, Xl)e (_ z+ i, Xl ) ^ W '*J 
I V3; *J d(-z+l, X xMz+hx^)tiOz + l,xT 3 ) 

*(<r(^),A,x) 

" e(-z + |,xiM* + j. xr l H3z + E xr 3 ) 
M2( } = e(-^i,x,xfr^,xr l xfexr J xoi-W) 

*(<7(tt),A,x) 

' cc-z+^xiMz+i.xrO^xr^+i.xr 3 )^- i.xr 3 ) 

Proposition 3.3.2. Res$ 2 ^(Z",/; A) a simple pole atz=\, i.e. A = ±/3 3 wAew 
xj = 1, i.e., X6 = 1 and Res i - Res$ 2 ^ (/*,/; A) = 0. 



Proof. All the local intertwining operators are holomorphic at z = \. So all the 
poles are contained in the normalizing factors. Therefore, 

Res^Ress^/'iA) 

= <*)|(*(P3, j&, x) - «(<**), x)]/(^3),/(^ 3 ) j 

+ (^((-,(,4/33,x)^K^),^3,x)y(^.3)/(l^)), 

with (*) and (**) being constants depending on xi- Here R(o(n\ |/3 3 , X ) = 

/?(P3,^3,x)/?(P6,^3,X)and/?(p 4 ,^3,x) = *(^X|A.x)^iA.X). Since 
P6X = X, by Lemma 3.2.4, R(p 6 , ±0 3f x) = id. Therefore, Res Res*, A(f,f ; A) = 0. 
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3.4. Calculation of Res Ss A(f,f; A). M(w, A, x) has a pole at S s if xs = 1 , U. X i X6 = 
1.0nS 5 , 

(A,^) = z-\, <A,0 2 V > = 2z, (A, ft) =z+ i 
(A,^>-z+2(A,«) = -z+5 

Lemma 3.4.1. 

ixiK(2r,Xi) /?(P3,A,x) 



A^(p 3 ,A,x) = 



«*+ f.Xi)e<2z+ l,X?)efc- i»XiM2z,x?)^+ J.xiVfc + J.Xi) 

3 



^5, ^xfK(^i,xiK(-^|,xr') 

lP4 ' **' C(2z+l,x?)^+f,X.)C(-^f,xr 1 ) 

/?(p4, A, x) 

e(2z,x?)e(z + 5,; 

A^(p 5 ,A,x) = 



e(2z,x?)e(z + 5,XiM^+ §,XiM-z + ixr') 
^+|,Xi^(-^+5,Xr 1 ) *(P5,A,X) 



^+ f.xi>e(-*+ §>xr')^ + §.xiW-*+ ixr 1 ) 

^ , . . x tt*-i,X.*(2z,X?*(-*+f,xr') 

= c(z + f,x,)^ + i,x?)^ + !,xr l ) 

/?(g(7T),A,x) 



e(z-i,xi)e(2z,x;M2 + ^XiMz + | > XiK-2 + !,xr 1 ) 

f 47 n a ^ ^ + i>xi)C(-^ + |»xr 1 ) 
^HT^^J-^+f.x^-x+f.xr') 

^(^( 4 f),A, X ) 



e (z+i,xiMz + §,xiM-z + i,xr') 

f.xr^eC-z+f, 



- dg&tf (^] 



PROPOSITION 3.4.2. /fx k «o* /riv/a/, Res$ 5 ^(/",/; A) /ias a s/m/?/<? po/e af A = fa, 
i.e., -when \ 2 \ = 1. Xi ^ ' ,/<s residue is given by 

Res^Res* ^(A/; A) = c^Rfa.faxWJSdh)) 

+ c 2 ( (tf(p 4 , ft, x) ♦ 04 , x)y034),/034)) , 

PROOF. All the local intertwining operators are holomorphic at z = ±. Therefore 
all the poles are contained in the normalizing factors. Our assertion follows from the 
straightforward computation. 
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PROPOSITION 3.4.3. If \ « trivial, Res Si A(f,f;A) has a triple pole at z = +. 
Moeglin-Waldspurger calculated the residue. 

3.5. Calculation o/Ress, A(f,f ; A). A/(w, A, x) has a pole at S 6 if X6 = 1 • On S 6 , 

(A,#) = 2-i (A,/3 2 v ) = 3z-i, <A,#) = 2z, 



<A,fl) = 3z+± (A,#) = z+i. 



Lemma 3.5.1. 

aAp4,A )X ) = 



«3z-S,X?K(2z,X?K(z+5.Xi) 



/?(P4,A > x) 

' ,(3z - i, x?)e(2z, X ?K3z + i, x ?)e(z + i, Xl ) 

^ «3*+|,XiK(* + S'Xi) *(fls,A,x) 

lP5 ' 4(3z+|,x;)^+|,X.M3z+|,x?Mz+i,xi) 

A^(p 6 ,A, X ) = /?(P6,A,x) 

A - fl*-i,Xi*(3z-j,xiX(2z,x}) 
1 ( X ' X) " C(z + !,X,)e(3z + |,x?)4(2, + l,X?) 

*(*(*), A, X ) 

* e(z- I,x,M3z- I,x;M2z,x?M3z + i,x?Kz+ £, X l) 

£(2z,xfe(3z+i,xfc(z+ 2 -,x.) /?(g(H A >x) 
£(2z + l, X ?)£(3z + f, X ^(z+ |,x.M2z,x?K3z+ i,x?Mz+ i,xO 

V V 3 /' '*J ^(z+|,xiMz+i,Xi) V V 3 V' 'M 

All the local intertwining operators are holomorphic at z = g,z = \,z— \ and so all 
the poles are contained in the normalizing factors. 

Proposition 3.5.2. lf\ is not trivial, Res^ A(f,f; A) has a simple pole at A = ^/3s, 
i.e., z = } if X ] = 1. Resi^ Res^ A(f,f; A) = 0. 

Proof. 

Res, ^Res* A(f,f; A) 

-*(M*J**)*KKTr>M>fi*V , fi*)] 
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where c\ , c 2 depend on \ i . But 

/?(p4,^ 5 ,x)=/?(p6,-j/33,x)/?(^(y),^5,x] 

*(o(xX j/5s, x) = R(ps, x)r(p2, ijfc, x) • 
Here <r(^)x = x and P6P1X = P\X and 

r(pi, i/3 5 ,x) = |ft,Pix)*(p6. |/3 3 ,Pix)*(pw ^/3s,x). 

By Lemma 3.2.4, /?(p 6 , - 5^3, x) and/?(p6, 5 ft, Pi x) are the identity. By the cocycle rela- 
tion of the normalized intertwining operators, R(p2, \(3s, x) is also the identity. Therefore, 
the residue is zero. 

Proposition 3.5.3. A(f,f\ A) has a simple pole at A = fa, i.e., z = \ if\\ = 1. 

0) Xi = 1. X? ^ 1- The residue is given by 

Res^ 4 Res* ^(f,/; A) = c(flM, Ufa, fa PaxWa) + *(p. , ft, x/Cfc))' 

(2) X? = I. Xi ^ 1. The residue is given by 

Res^Res^/iA) = Ci^^A^+^'X^.x))^^^)) 

+ o ^("(y).fc. x)/l04XA02)j , 

Proof. It follows from Lemma 3.5. 1 and straightforward computation. 

Proposition 3.5.4. If \ is trivial, Res* A{f,f \ A) Aas a fr/p/e po/e a/ A = /? 4 . 
Moeglin-Waldspurger calculated the residue. 

Proposition 3.5.5. Ifx *■ /r/v/ a /, Res* /!(/',/'; A) /las a swip/e pole at A = p fl , «.«., 
z = |. 77ie residue is given by 

c(F)£(-) 1 \ 
Res„ Res* A(f,f; A) = (/?(<t(7t), Pb , X^X/Cp*)) . 

This gives the constant. 

3.6. Conclusion. Let J(x) be the subspaceof I& S (G{F)\G(\)) {W which is ^-finite 
and /^oo-invariant. 
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3.6.1. x is trivial. We can see from the above calculation (Propositions 3.2.3, 3.3.2, 
3.4.3, 3.5.4, 3.5.5) that Moeglin and Waldspurger obtained all the residual spectrum at- 
tached to the trivial character of the torus. We summarize their results; J( 1 ) is isomorphic 
to the sum of the trivial representation and the image ( 1 + \E)R(pi, /3 2 )/(/3 2 )/, where I(fh)f 
is the /koo-invariant subspace of /(/3 2 ) and E = ® v E v . Here E v is defined as follows: Let 
R(ps, P2A)\s t be the restriction of /?(p 5 , p 2 A) to S\ . It is holomorphic at (i 2 - Let E v be the 
value of /?(p 5 ,P2A)| 5| at ft- 

Then R v (p2, j3 2 )/ v (/? 2 ) = tti v © 7r 2v , where ir w is spherical and 



where ttq is the trivial representation. 

3.6.2. x is non-trivial. From Propositions 3.2.2, 3.4.2 and 3.5.3, only the following 
characters contribute to the residual spectrum: 

(1) x:xi»i,xi-i.x«*i 

(2) X ' = P6X:X\ 2 =UX\¥UX' 6 = X\ 

(3) x" = P.P6X:x; ,2 =Ux;'^l,X^l 

(4) X:X 3 = 1,X.^1,X6= 1 

Under the identification M, ~ GL 2 , where M\ is the Levi subgroup of P\ , the above 
characters are given by x = x(p-» » / )» where n, v are grossencharacters of F: 

(1) X = X(P,vlH = v,H 2 = 1 

(2) p 6 x = xiP>*\ = 1,1/^1 

(3) P1P6X = x(P-,"),P 2 = Up ¥ Uv = 1 

(4) x = x(wlS = hn¥\,v = n 2 

CASEl. x=Xi = Uxl= 1»X6^ 1 
From Proposition 3.2.2, 

Res^ Res Sl A(f,f r ; A) = c x {f(p 2 ),R{p2,Pi,p2X) 

^(P5,"&", x/Ofc) + <*2*(*(f) . ft, P3x)T (ft) + ^(p^yX^wj, 

where c,,c 2 ,c 3 are constants and /?(p 5 , "ft", x) = <8>v/?(P5, "A", Xv) and R(p s , "ft", 
Xv) is the value at (3 2 of the restriction R(p s , A, Xv)U, . It is an isomorphism from I(0 2 , x) 
to/(/3 2 ,p 5 x). 

Here we recall the inner product formula (3.1) and our short-hand notation. Note that 
G W(X>X),P3 e »F(x,P6X)anda( 2 f) € JF(x, Pi p 6 X>- Therefore, the above /"s 
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are all in different spaces. Since /?(ff(f ),04,P3x) and /?(p 6 ,04, yX) are intertwining 
operators and /?(p 5 , "ft", x) is an isomorphism, J(\) is isomorphic to the image 

R(p2,02,P2XW2,p2X)f = ® R V (p2,02,P2Xv)Wl,P2Xv). 

v<oo 

We already know from Moeglin-Waldspurger that if Xv is trivial, 

R V (p2,02,p2Xv)W2,P2Xv) = TTiv ©7r 2v . 

Suppose x v is not trivial. Then \v <8> exp(0 2 , // B ()) is a regular character of 7\ So we can 
apply Rodier's result as follows: By the cocycle relation, 

Rv(P2,P2,P2Xv) = Rv(P5,-02,P5Xv)R V {°(*),P2,P2Xv)- 

From Rodier [R, Cor 3 in p417], /? v (ct(7t),0 2 ,p 2 Xv)/v(02,P2Xv) and the unique irre- 
ducible subrepresentation of / v (-02,p2Xv) bave the same Jacquet module. There- 
fore, R v (<j(7r),02,p2Xv)l v (02,P2Xv) is the unique irreducible subrepresentation of 
h(-p2,P2Xv). Since /? v (p 5 , -0 2 ,PsXv) is an isomorphism, /?v(p2,02,P2Xv)/v(02,P2Xv) 
is the unique irreducible subrepresentation of I(—(h,Xv)- 

Let J v = {ttiv, ?T2v}. If Xv is not trivial, we take 7r 2v = 0. Let S be a finite set of finite 
places and 7T 5 = 7r ( „ ® (g^s 7r 2v . Then 

Ax) = 0^. 

There is no condition on S. 

Case 2. x' = PsX-X? = hx\ ¥ Ux' 6 = x\ 
From Proposition 3.4.2 and the adjoint formula (3.2), 

Res^ Res$ 5 A(f \f\ A) 

= C l (f(P 4 ),R(P3,02,P3XY(02)) 

+ C 2 ^4), /?(P4, 04, P4X')) (AA) + JKPI , ft, X Y (04)) . 

Recall the inner product formula (3.1). We note that p 4 € W(p 6 x,PiP6X) since 
P4P6X = P\P6X and e W(p 6 X,P6X), P3 6 ^(p 6 X,X)- Therefore, the above /"s 
are all in different spaces. Here 

*(P 4 ,04,P4XV(/34,P4X') = (g)^(P4,04,P4Xvyv(04,PvXv)- 

v 

Under the identification Mi ~ GL 2 , P6X = X(M* v )i P = 1. 1/2 = U * 7^ 1. By 
inducing in stages, 7(0 4 , P4X v) = Ind ?, exp(0 4 , ()) ® IndJ o Lj p 4 xt» wnere #o is a Borel 
subgroup of GL 2 . ir v - lnd£ o L2 p 4 x v is irreducible. Therefore, fl v (p 4 , 0 4 , P4Xv)A>(04, PvX'v) 
is the Langlands' quotient of Indp, exp(04, ///>, ()) ® tt v . In particular, it is irreducible. 

We have P3 = P1P4P6 and p 4 \' = X- So 

/?(P3,02,Xv) = ^(Pl,-04,p4P6Xv)^(P4,04,P6XvV?(P6,02,Xv). 
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If \v is not trivial, then R(p lt — 0 4 ,p 4 p6Xv) and /?(p 6 ,02,Xv) are isomorphisms. There- 
fore, the image of R(p),02, Xv) is irreducible. 

If Xv is trivial, R(pe,02,Xv) is not an isomorphism and we proceed as follows: 



Since p 3 = Pi<Ky), R(Pi,Pi,Xv) = R(p\,-pA,<T( 2 flXv)R(<r(%),p2,Xv)- As in 
Lemma 3.2.4, we can show that R(pi, -/? 4 ,o(y)Xv) is the identity. Also from [M-Wl, 
equation (15)], R v (p 6 , -02)(*v(P2,ft) ~ EMPi,fa)) = 0. From (3.3), £/ = -2/ for 
/ € 7r 2v and so R v (p 6 , -foV = 0 for/ € 7r 2v . Therefore R v (p 6 , -^Ryifii^iWi) is ir- 
reducible. Since p 6 p 2 = <*(y ), the image of /? v (o r (y ), 02 ) is irreducible. It is isomorphic 
to the image fl v (p 4 , &)/(#»). 

Therefore, J(\') »s isomorphic to the image R(p4,P4,P4x'Wa,Pax')- Since p 6 p 4 = 
ft. -Ax') is isomorphic to none other than <g> v 7r, v in Case 1. 

Case 3. x " = Pit** xf = 1. xf * 1. X? » 1 
From Proposition 3.5.3 and the adjoint formula (3.2), 

Re S/j4 Res* A) = c, (Aft), «0M,ft, P4x")(f(M + *(p. , 04, xV<fc)) 



We proceed in the same way as Case 2. J(x") is isomorphic to the image 
R(P4, (3a, Pax"V(Pa, P4X")- ^ is irreducible and it is isomorphic to the one in Case 2. 

Case 4. x:*1 = 1. X? ^ I. X6 = 1 
From Proposition 3.5.3, 

residue = Res^ Rcs 5 „ A{f y f \ A) 

= c(f(J3 4 ), R(Pa, Pa, P4X)(/ V (04) + R(P\ , Pa, X.Y(Pa)). 

So J(x) is isomorphic to the image /?(p 4 ,/3 4 ,p 4 x)/(/54,P4X)- By inducing in stages, 
1(04, paXv) = Indp ( exp(/? 4 , ///>,()) <S> Ind^ 2 p 4 x v , where # 0 is a Borel subgroup of GL 2 . 
7r v = IndJ o L2 p 4 Xv is irreducible. Therefore, R v (p4, P4, p4Xv)Iv(04, PvXv) is the Langlands' 
quotient of Ind^ exp(/3 4 , ///>,()) <8> ttv- So J(x) is irreducible. 

We have proved 

THEOREM 3.6. 1 . LetJ( X ) be the subspaceofL 2 dia (G(F)\G(\)) (Tx) , which is K-finite, 

Koo-invariant. Then the K-finite, Koo-invariant subspace of L\ % (G(F)\G(K)) T is the di- 
rect sum of the following space: 

(1) J(l) = 7T 0 0 ©5,card(5)/i where tt 5 = ®v£ S 7r, v <g> ®v€5 7T2v. tti v is spherical. 

(2) J(x) = ©stt 5 . wAerexi = I. xl = I X6 7^ » ^ = ®^ S 7r, v ® ® v6S 7r 2v . 7/" X6v 
« no/ trivial, we set n2v = 0. 

(3) J(x) = the Langlands 'quotient of Ind£ exp(/3 4 ,// P| ()) <8>(Ind° L2 x), where x 3 = 
1. Xi ^ I. X6 = I. 

3.7. Arthur Parameters. In this section, we give the Arthur parameters for the residual 
spectrum of L 2 dis (G(F)\G^j) r We say that a unipotent element u is distinguished if all 
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maximal tori of Cent(«, G) are contained in the center of G°, the connected component 
of the identity. This is equivalent to the fact that the unipotent orbit O of u does not meet 
any proper Levi subgroup of G (Spaltenstein [Sp, p67]). (i.e., if L is a Levi subgroup of 
a parabolic subgroup of G and u G L for a u e 0, then L° = G°.) 

JACOBSON-MOROZOV Theorem. Suppose uisa unipotent element in a semi-simple 
algebraic group G Then there exists a homomorphism <f>: SL2 •— > G such that 

♦(J !)- 

LEMMA ([B-V, PROP. 2.4]). Let u be a unipotent element and <f>: SL 2 »-» G be a 
homomorphism such that 4>{^ \] = u - Let S 4> = Cent(im <f>, G) C S u = Cent(w, G) 
and W be the unipotent radical of S u . Then 

(1) S u = S 4t U u ,a semi-direct product. is reductive. 

(2) The inclusion C S„ induces an isomorphism between S^/SJZc andS u /S° u Z G . 

Let F be a number field and let W F be the global Weil group of F. For G a split 
group of type G 2 , we can take the dual group G* = G 2 (C). An Arthur parameter is a 
homomorphism 

xp: W F x SL 2 (C) »-» G*, 

defined modulo conjugacy, with the following properties: (The usual definition of Arthur 
parameter uses Langlands' hypothetical group L F . But since we are only dealing with 
principal series, Wp is enough.) 

(1) ip(W F ) is bounded and included in the set of semi-simple elements of G*. 

(2) The restriction of i/; to SL 2 (C) is algebraic. 
Let S+ = Cent(im V, G*) and 

Now we recall Moeglin's reformulation of Arthur's conjecture ([M3]): For each place 
v of F, we have local Arthur parameters xp v = V| W Fy x SL 2 (C), as well as S^, C^. It 
is a part of local Arthur's conjecture that for each irreducible character t/ v of C^, there 
exists an irreducible representation tt(t/; v , tj v ). For each v, let n 0v be the set of w(Vv. Vv)- 

We define the global Arthur packet IT^ = L 2 (G(F)\G(A)) , to be the set of irreducible 
representations 7r = ® v tt v of G(A) such that for each v, ix v belongs to n^ v and for almost 
all v, 7T V is spherical. 

Arthur's conjecture (Global). (1) The representations in the packet corre- 
sponding to V may occur in the discrete spectrum if and only if Q, is finite, i.e., SJ = 1 . 
We call such an Arthur parameter elliptic. 

(2) For an elliptic Arthur parameter V\ any ?r € F^ occurs discretely in L 2 (G(F)\G{^) 
if and only if 

(3.4) e n^v^o, 

*ec<, v 
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where 7r = ® v 7r(^ v , *?v)> x = (x v ). 

Remark 3.7.1. If Q, is abelian, then the above condition is equivalent to: the charac- 
ter n v 7; v |Q, of Q is trivial. This is what happens in split classical groups [M 1 ]. However, 
it is not true in our G 2 case since is not abelian as we see below. It is S 3 , the symmetric 
group on 3 letters. 

Let rires, be the subset of n v „, parametrizing the local components of the residual 
spectrum. We will find Yl^ and verify (3.4) for a representation 7r = ® v 7r v , 7t v G n reSy 
for all v, 7r v spherical for almost all v. 

Remark 3.7.2. A representation in n^ v but not in n reSi will appear as a local com- 
ponent of a cuspidal automorphic representation. Suppose we know the local packet n^ v 
completely. Then it is a very difficult problem to determine when a representation n e Uj, 
is a cuspidal representation. Moeglin [M5] has a partial result on that in the case of split 
classical groups. 

3.7.1.x trivial. 

The Arthur parameter is given by 

0: Wp x SL 2 (C) G 2 (C), 

where xp\w F = 1 and \\>: SL 2 (C) »— * G 2 (C) is given by a unipotent orbit of G 2 (C). In order 
that tp be elliptic, the unipotent orbit has to be distinguished. There are two distinguished 
unipotent orbits of G 2 (C), namely, G 2 (C) and G 2 (ai) ([Ca, p401]). 

Case 1 . The unipotent orbit G 2 (C). 

The unipotent orbit G 2 (C) gives the constant which corresponds to the residue 

Res Pfl Res 56 ^(/",/;A). 
Case 2. The unipotent orbit G 2 (ai ). 

If V is determined by the unipotent orbit G 2 (a ( ), then C$ = = S 3 , the symmetric 
group on 3 letters. There are 3 irreducible characters of S 3 , namely, V> 3 , V21 and i/> m . 
Here t/>m is the sign character of 5 3 . They are class functions and the character table is 
given by 





03 


^21 


0111 


c, 


1 


2 


1 


c 2 


1 


0 


-1 


c 3 


1 


-1 


1 



Character table of 



Here C\ t C 2 and C 3 are the conjugacy classes in S 3 , namely, C\ = {1}, C 2 = 
{(1,2),(1,3),(2,3)}, C 3 = {(1,2,3),(1,3,2)}. From Section 3.6.1, we know that 
Kv(P2, = 7Ti v © 7T 2v , where tt, v is spherical. We attach tt, v to 0 3 and tt 2v to 
rp 2 i ■ Therefore, in this case, n rcSi = {tt\ v , 7r 2v }. 
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Then 7r = <S) v ^s^\v ® <8>vg5 7T2v appears in Ll is (G(F)\G(k)) if and only if 

®^3(l)®®^i(l) + 2®V3((l,2,3))<8>®^2i((1.2,3)) 
(3.5) * s *s * s 

i.e., s ^ 1, where s = \S\. This coincides with Moeglin-Waldspurger [M-Wl, Ap- 
pendix III]: In Moeglin-Waldspurger, there is an operator E v which acts on it\ v and 7r 2v 
(See Section 3.6.1). Since we attached tv\ v to t/>3 and 7r 2v to ^21, £V acts on the irre- 
ducible characters of S3 as follows: £ v (03) = 1 and E v (ip2i) = —2. Then we can see that 
£ v (n) = 7/(1)77(0 23)) for 7/ = V>3, t/>2i. Therefore we can write (3.5) as follows: 

2 ® 2, 3)) ® ® V21 (0. 2, 3))(l + x ® ® Mfo) 9 (8) 1 )) * 0 

/.e.,7r = ® v tt v appears in Ll s (G(F)\G{fij) if and only if(l+±£>r ^ 0, where £ = ® E v . 

Remark 3.7.3. According to Arthur's local conjecture, the sign character Vm 
should give an irreducible representation which is a local component of a cuspidal auto- 
morphic representation. We do not know what it is. 

Remark 3.7.4. For O a distinguished unipotent orbit, let A{u) = C(k)/C(u)°, where 
u € O and C(u) is the centralizer of u. Let Springer(0) be the set of irreducible characters 
of A(u) which are in the image of the Springer correspondence which is an injective map 
from the set of irreducible characters of W into the set of pairs (O, rj), where O is a 
unipotent orbit and 77 is an irreducible character of A(u) = C(u)/C(u)°, where u € O. 
We note that by [Ca, p427], Springer(G 2 (fli )) = {^3, V>2i } in G 2 (C). Therefore the local 
component n rcSv of the residual spectrum attached to the trivial character of the torus is 
parametrized by Springer(C/2(ai )) . Moeglin [M 1 ] showed that for split classical groups, 
the residual spectrum attached to the trivial character of the torus is parametrized by 
distinguished unipotent orbits O and Springer(0). In other words, if the Arthur parameter 
tl> is given by the distinguished unipotent orbit O, then 11^ = Springer(0) and the 
multiplicity formula (3.4) holds. 

Therefore we believe that the same thing would happen for all split groups. We state 
this as follows: 

CONJECTURE. Let G be a split group over a number field F and T be a maximal 
torus of G. Then the residual spectrum attached to the trivial character of T(\)/T(F) 
is parametrized by distinguished unipotent orbits O of G*(C), the L-group of G and 
Springer(0). More precisely, if the Arthur parameter 0: SL 2 (C) 1— > G*(C) is given by 
the distinguished unipotent orbit O, then 11^ = Springer(0) and the multiplicity for- 
mula (3.4) holds. 

We give an example of this conjecture in the case of split exceptional group F 4 and we 
hope to settle this example in the near future: Suppose the Arthur parameter 0: SL 2 (C) h-» 
F 4 (C) is given by the distinguished unipotent orbit F 4 (a 3 ). By [Ca, p40 1 ], A(u) = S 4 , the 
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symmetric group on 4 letters. There are 5 irreducible characters of S4, namely, 0 4 = I, 

031, 022, ton and tolll- By [Ca, p428], Springer(F 4 (a 3 )) = {to,toi»to2,tou }• The 
character table of S 4 is given by 





c, 


c 2 


c 3 


c 4 




to 


1 


1 


1 


1 


1 


01111 


1 


1 


1 


-1 


-1 


ton 


2 


2 


-1 


0 


0 


to2 


3 


-1 


0 


1 


-1 


to. 


3 


-1 


0 


-1 


1 



Character table of S 4 



Here Q,i = 1 , . . . , 5, are the conjugacy classes in S4, with representatives 1,(1 2)(34), 
(123), (12) and (1234), respectively: |C, | = 1, |C 2 | = 3, |C 3 | = 8, |C 4 | = 6 and |C 5 | = 
6. According to the conjecture, there will be 4 irreducible representations 7T| V ,. . . ,7r 4v 
attached to 0 4 , 0 2 n, to2 and 03 1, respectively. We divide Springer^^)) as follows: 

Springer(F 4 (fl3)) = rii U n 2 U n 3 , where rii = {0 4 , 0 2 n }, n 2 = {to, 022 } and n 3 = 
{04, 03i }• The residual spectrum factors through n„ i.e., it is the set of all -n = <g> 7r v such 
that there exists 1, n v £ n, for all v. Let S be a finite set of finite places and s = \S\ . If it = 
<8)v^5 7riw®<8>v€s7r 2v , it appears in 4 S (G(F)\G(A)) if and only if2 J +3(2 , )+8(-l) 1 ^ 0, 
i.e., s ^ 1. If 7r = glvfs^iv ® Qvesniv, it appears in Ll is (G(F)\G(k)) if and only if 
V + 3(-l)* + 6 + 6(-l)» ^ 0, i.e., 5 =^ 1. If 7T = <8>^ s 7Ti v <g> Q^s**, it appears in 
Ll s (G{F)\G{hj) >f and only if 3* + 3(-l)* + 6(-iy + 6 ^ 0, i.e., s?\. 

3.7.2. x non-trivial. In order to find Arthur parameters for non-trivial characters, we 
have to look for endoscopic groups of G2(C), since Arthur parameters will factor through 
the endoscopic groups. 

There are two equivalence classes of proper cuspidal endoscopic groups of G 2 (C), that 
is, SL 3 (C) and SL 2 (C) x SL 2 (C)/{±1} ([Al, p30]). They are given as follows: Under 
the identification M\ ~ GL 2 , by (2.1), 0%(t) = diag(/,r). Then by [Ca, p93], C(/%(uj)), 
the centralizer of /3 4 (v) in (^(C), where u? = 1, u) ^ 1, is reductive and its root system 
is O, = {±/3 u ±03,±Ps}, i.e., C(fi(u})) ~ SL 3 . The other one is C(/^(-l)). By [Ca, 
p93], its root system is <D, = {±0 x ,±fa}, i.e., C(f3^(-\)) ~ SL 2 (C) x SL 2 (C)/{±1}. 

The center of SL 3 (C) is Z 3 = {w/j.w 3 = 1} ~ Z 3 and the center of SL 2 (C) x 
SL 2 (C)/{±1} is Z 2 = {±/ 2 }. Moreover, S 3 = Z 3 x Z 2 . 

Case 1 . The conjugacy class of \: xi = 1, \l = 1, X6 ^ 1 
Under the identification M\ ~ GL 2 , x = x(M»/A M 2 = U H ^ 1, where /i is a 
grossencharacterof F. We have the embedding SL 3 (C) C (7 2 (C). 
The Arthur parameter factors through SL 3 (C): 

0: W F x SL 2 (C) ^ SL 3 (C) ^ ^(C). 

0|if>: w >— ♦ n(w) and xp: SL 2 (C) 1— ♦ SL 3 (C) is determined by the principal 

unipotent orbit of SL 3 (C). Here we note that under the embedding SL 3 (C) (^(C), 
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the principal unipotent orbit of SL3(C) corresponds to the distinguished unipotent orbit 

G 2 (ai) in G 2 (C) ([Ca, p401]). Then 5^ = Z 2 , C 0 = Z 2 and Q, v = Z 2 if p. v is not trivial. 
Q, v = S3 if p v is trivial. 

If p v is trivial, then n rcSv = {7r )v ,, 7r 2v }. If p v is not trivial, then n reSv = {7r lv }. Then 
any n = (^v^s^iv <8> <S)ves^2v appears in Z^(G(F)\G(A)) since 

(g) t/> 3 ( 1 ) ® <g> fci 0 ) + ® te(r) <8> (g) V21 (t) ^ 0, 

v£S v€5 v£S v€5 

where t is the non-trivial element in Z 2 . 

Case 2. The conjugacy class of x: x] = 1, Xi ^ 1, X6 = 1 
Under the identification, A/, ~ GL 2 , x = x(/*, = *t/* ^ 1. v = /i 2 . The Arthur 
parameter factors through SL 2 (C) x SL 2 (C)/{±1 }: 

V>: »> x SL 2 (C) h-* SL 2 (C) x SL 2 (C)/{±1} ^ G 2 (C), 

where V^rvv ^-i^j X [J J] and V:SL 2 (C) SL 2 (C) x SL 2 (C) 

is determined by the principal unipotent orbits of SL 2 (C). We note that under the em- 
bedding SL 2 (C) x SL 2 (C)/{±1} G^C), the principal unipotent orbit of SL 2 (C) x 
SL 2 (C)/{±l}correspondstotheunipotentorbit>I| inG 2 (C)([Ca,p401]).Then^ = Z 2 , 
= Z 2 for p v non-trivial and Q v = 1 for p v trivial. In this case, consists of the 
Langlands' quotient which corresponds to the trivial character of Q,, = Z 2 . Therefore 
fires consists of one element. 

Remark 3.7.5. Arthur associated to an Arthur parameter xp y an associated Lang- 
lands' parameter <f>^ and conjectured that we could enlarge the L-packet n^ v to n^„. We 
note that in each of our cases, the associated Langlands' L-packet consists of only one 
clement. 

4. Decomposition of L 2 dh (G(F)\G(h)) ^ . We have 

a* Pi =X(M x )®R=R0 A ,a Pt =R0 w 4 

p Px is the half sum of roots generating ti\ . Then p Pi = |/3 4 . 

Let a = 04 and identify s e C with sa 6 a£. Let 7r = ® 7r v be a cusp form on 
M\ = GL 2 . Given a AT-finite function ip in the space of tt, we shall extend ip to a function 
Cp on G and set 

<J>, (g) = <p(g) exp<5 + p Pl , H P] (g)) . 
Define an Eisenstein series 

-reP,(f)\G(F) 

It is known that E(s, <p,g,p\) converges for Re(s) » 0 and extends to a meromorphic 
function of s in C, with a finite number of poles in the plane Re(s) > 0, all simple and 
on the real axis. 
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It is also known that Lj ls ((7(F)\(/(A)) A/ is spanned by the residues of the Eisenstein 
series for Re(s) > 0. We know that the poles of the Eisenstein series coincide with those 
of its constant terms. So it is enough to consider the constant term along P { , which is 

E 0 (s,<p,g,Pi) = Y, M(s,*,w)f(g) 
wen 

where Q = {l,p 6 pip 6 piP6} and 
where 

k= n u a , 

or>0 
w'ar<0 

U a is the one parameter unipotent subgroup and / e I{s, tt) = Ind^ tt <g> exp(s, H Pl ()) . 

We note that for each s, the representation of G(A) on the space of O s is equivalent to 
/(*, tt). 

Then 

M(s, tt, w) = <g) M(s, tt v , w), M(s, 7r v , w)/" v (g) = / r /v(w~ 1 wg) </« 

where / = <8>/ v ,/v is the unique Affixed function normalized by f v (e v ) = 1 for almost 
all v. 

Let L M\ = GL 2 (C) be the L-group of Mi . Denote by r the adjoint action of L M { on 
the Lie algebra 1 n i of L N\ , the L-group of N\ . 
Then 

r = n © r 2 
n = r\, r 2 = A 2 p2 

where r% = r 3 <8> (A 2 p 2 ) _l is the adjoint cube representation of GL 2 (C) (See [S4]). Here 
r 3 is the symmetric cube representation of GL 2 (C) and p 2 is the standard representation 
ofGL 2 (C). 

Then it is well-known ([S3]) that for w = P6P1P6P1P6 

ves V £s M 1 s,tt, r\ )Ls( 1 + 2s, tt, r 2 ) 

where 5 is a finite set of places of F, including all the archimedean places such that for 
every v £ 5", tt v is a class 1 representation and if/ = ® v f v , for v ^ 5,/ v is the unique 
Affixed function normalized byf v (e v ) = l./ v is the AT v -fixed function in the space of 

/(-s, w{tt v )). 

Finally, £s(s,*,r<) = n^ 5 L(i,7r v ,r,), where L(s,7r v ,r t ) is the local Langlands' L- 
function attached to jt v , r,. 
(1) Analysis of L s (s, tt, r\ ) 
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We know ([S4]) that Ls(s, jt, r\ ) is absolutely convergent for Re(s) > 1 and hence has 
no zero there. It is expected ([S4, Bu-G-H, Ik]) that the completed L-function L(s, n,r\) 
has a pole for Re(s) > 0 if and only if s = 1 , wj as l , u w ^ 1 and -k is the monomial rep- 
resentation corresponding to the quadratic character u;*, where u) w is the central character 
of jr. We assume this fact. 

Remark. Ikeda [Ik] calculated the poles of the Rankin triple L-function L(s, 7r® 7T<g> 
7r) for 7r cuspidal representation of GL 2 . It is related to the symmetric cube L-function of 
7r as follows: 

L(S, IT ® 7T <g> 7T) = Z,(s, 7T, r 3 )(Z,(.S, 7T ® u;*)) . 

The symmetric cube I-function is given by 

£(f,*,r*) = £(j, 

L(s, 7r, r 3 ) has a pole at s = 1 when i4 = 1 ano< ^ 1 and so L(s, n, rf) has a pole when 
u£ = 1 and a;* ^ I. 1 

(2) Analysis of Z,s(s,7r,r 2 ) 
For v £ 5, 

L(5,7r v ,r 2 ) = 1(5,00 = (1 - ^.(^T 1 

so Lsis,TT,r 2 ) is the (partial) Hecke L-function. It has no zero for Re(s) > 1 . The com- 
pleted L-function L(s, ir, r 2 ) has a pole for Re(s) > 0 if and only if s = 1 , = 1 . 

(3) Analysis of M(s, tt v , w) for v € S. 

For tt v tempered, the local factors L(s,-n Vt ri) and M(s,ir v ,w) are holomorphic for 
Re(s) > 0. We show that for any v e 5, 

L(S, 7T V , n )" ' £(2S, 7T V , r 2 )" ' Af(s, 7T V , w) 

is holomorphic. It is enough to show it for 7r v complementary series. We follow [Ki]. 
Under the identification M\ ~ GL 2 , by (2.1), for tt v = tt(/x| | r ,/x| |- r ), 0 < r < \, 
complementary series of GL 2 , 

Ind£ 7r v ®exp((5d,// Pl 0» = Indg x(M^) ® exp((A, //*())), 

where A = (2r)/? 3 + (s — 3r)04. From this we have our assertion. 

Now we assume that r < \. Right now the best known result is r < j due to Shahidi 
[S3]. Then A is in the positive Weyl chamber for s = \ and s = 1 and we have 

LEMMA 4.1. For each v, the images ofM{\, ir v , w) and M(l,n v ,w) are irreducible. 



Thanks to F. Shahidi who pointed this out. 
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Conclusion. E(s, <f,g,P\) has a pole in the half plane Re(s) > 0 if and only if 

(1) w,= l,*=£,I(| t ir,iS)^0, 

(2) u% = 1, us* ^ 1,5= 1, 7r monomial representation attached to cj t . 

Let J\(ir v ) be the image of M(j,tt v ,w) and J 2 (^ v ), the image of M(l,7r v ,w). They 
are the unique irreducible quotients of I(\,ir v ) and /(l,7r v ), respectively. LetJ\(n) = 
(8» v Ji(7T v .) andJ 2 (n) = <g> v J 2 (ir v ). We have proved 

Theorem 4.2. 

where ir\ runs over cuspidal representations of GL 2 with trivial central characters and 
L{\, 7T, r?) ^ 0 a«c/ 7T 2 runs over monomial representations. 

5. Decomposition of /4(G(F)\<7(A)) . 

In this case a* Pi = X{M 2 ) ® R = R0 3 , Q/> 2 = R$(, p/» 2 = § #»• 
Let or = /3 3 and identify 5 € C with sar e a£. In this case, for 7r cuspidal representation 
of GL 2 , the constant term of Eisenstein series is given by 

Eo(s,<P,g,P 2 )= £ M(s,it,wY(g) 
wen 

where O = {l,pip6PiP6Pi}. 

The adjoint action r of L M 2 on L n 2 is given as 

r = n 0 r 2 0 r 3 

= p 2 ,r 2 = A 2 p 2 ,r 3 = p 2 ® A 2 p 2 . 

Therefore for w = pip 6 pip 6 pi, 

7T, wy = (g) A/(s, 7r v , wy v ® <g)/ v 

x ^5(5, 7T, n )L$(2.s, 7T, r 2 )Z, 5 (3s, tt, r 3 ) 
Z<s(s + 1 , 7r, ri )L s (2s + 1 , 7r, r 2 )Z-s(3.s + 1 , jr, r 3 ) 

where 5 is the same as in the case L^ S (C(F)\G(A)) M . 
Here 

L(s, 7r v , r\) = L(s, 7r v ), the standard L-function for GL 2 . 
L(s, 7r v , r 3 ) = Z,(s, 7r v ® a; Wv ), twisted by the central character. 
L(s, 7r v , r 2 ) = L(s, uj %v ), Hecke L-function. 

We know that L s (s, it <g> 0) is absolutely convergent for Res > 1 for any grossencharacter 
9. So it has no zero there. We know also that the completed L-function L(s, ir <8> 9) is 
entire for any 0. Under the identification M 2 ~ GL 2 , by (2.2), for 7r v = 7r(p| | r ,p| |" r ) 
complementary series of GL 2 , 

Ind% tt v (8) exp((sd, %0» = Indg X C*. *0 ® exp« A, // fl ()», 
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where A = (s - 3r)/? 3 + 6r(l 4 . Therefore, for any v e S, 

3 

11 L(is,n v ,n) 'A/(s,7r v ,w) 
1=1 

is holomorphic. Also if we assume r < A is in the positive Weyl chamber and the 
image of M{\, 7r v , w) is irreducible. Therefore, E(s, (p,g,P2) has a pole in the half plane 
Res > 0 if and only if u> n = 1, s = 5 andL(j,7r,ri) ^ 0. 

Let J(7r v ) be the image of M(\, 7r v , w) andJ(7r) = ® v J(n v ). Then we have 

Theorem 5.1. 

4 S (C(F)\G(A)) =07(tt) 

IT 

where n runs over cuspidal representations of GL 2 with trivial central characters and 
Z.(±,7r)^0. 

Remark 5.1. The referee suggested the problem of finding a connection between 
Arthur's conjecture and the non-vanishing of L-functions at s = \. Arthur [Al ] did it for 
the group PSp 4 . It would be interesting to do so in the above case. 
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COMPACTNESS OF A LOCALLY COMPACT GROUP G 
AND GEOMETRIC PROPERTIES OF A P (G) 

TIANXUAN MIAO 



Abstract. Let G be a locally compact topological group. A number of charac- 
terizations are given of the class of compact groups in terms of the geometric prop- 
erties such as Radon-Nikodym property, Dunford-Pettis property and Schur property 
of A p {G), and the properties of the multiplication operator on PF p (G). We extend and 
improve several results of Lau and Olger [17] to A P (G) and B p (G) for arbitrary p. 

1 . Introduction. Let G be a locally compact topological group. Let A(G) and B(G) 
be the Fourier and the Fourier-Stieltjes algebras of G, respectively. Taylor [22] showed 
that A(G) has the Radon-Nikodym property if and only if G G [AR] and B(G) has the 
Radon-Nikodym property if and only if G € [AU]. Later, Lau and Olger [17] studied 
extensively the geometric properties of A{G) and B{G). The main machinery for them is 
C* -algebraic since, as is well known, the predual of B(G) is the group C* -algebra C*(G) 
and the dual of A(G) is the von Neumann algebra VN(G). In this paper, we will establish 
several sufficient conditions for G to be compact. We point out that the C*-algebraic 
method, which worked for p = 2, does not work for arbitrary p since A P (G) and B P (G) 
are usually not related to any C* -algebra. As we know, A 2 (G) = A{G) for an arbitrary 
locally compact group G, and if G is amenable then B 2 (G) = B(G) and PF 2 {G) = C*(G), 
the group C* -algebra of G. We extend and improve several results of Lau and Olger [17] 
to A P (G) and B P (G) for 1 < p < oo (see Lau and Olger [17], page 321-322). We prove 
in this paper, among other things, the following: for any locally compact group G, the 
following are equivalent 

(a) G is compact; 

(b) G is amenable and each functional <p in B P {G) is almost periodic on PF P {G); 

(c) Each functional <j> in A p (G) is almost periodic on PF P (G); 

(d) There exists a nonzero functional <f> in A P {G) which is almost periodic on PF P (G)\ 

(e) For each/ in PF p (G), the multiplication operator rf. PF P (G) PF p (G) (defined 
by T/(g) = f ■ g for all g e PF p (G)) is compact; 

(f) G is amenable and for each / in PF p (G), the multiplication operator 7/: PF P (G) — » 
PF p {G) is weakly compact; 
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(g) G is amenable and the Banach algebra PF P (G) is an ideal of its second dual under 
either Arens product; 

(h) For any a e A P (G), or equivalently for a nonzero a G A p (G), { x a : x € G} is a 
compact subset of A P (G). 

We also prove that each of the following conditions is sufficient for G to be compact: 

(i) The space A P (G) has the Schur property; 

(j) G is amenable and the space B P {G) has RNP and DPP; 

(k) The algebra PF p (G) does not contain an isomorphic copy of 1 1 and has the DPP; 

(1) Every bounded linear operator T: PF P {G) —* B p {G) is compact. 
The main tools we use to prove our results are Lemma 3. 1 , Lemma 4.5 and Lemma 5. 1 
in which various sequences approaching "infinity" will be considered. We organize this 
paper as follows. In Section 2, we collect the notations and definitions that will be used 
in this paper. In Section 3 we give various sufficient conditions for a locally compact 
group G to be compact in terms of some geometric properties of A P (G). In Section 4, 
by developing the technique used in Section 3 we characterize the compact groups in 
terms of almost periodic functional on PF P (G) and PM P (G). In Section 5 we give some 
results relating the properties of the multiplication operator rf. PF P {G) —> PF P {G) to the 
properties of the group G. 

2. Preliminaries and Some Notations. Let G be a locally compact group with a 
fixed left Haar measure A. If G is compact, we assume X(G) = 1. Let LP(G) be the 
associated Lebesgue spaces (1 < p < oo). For any /: G — > C, and x € G, the left 
[right] translation off by jc is defined by Jiy) = f(xy) [f x (y) = f(yx)], and / is defined 
by /(y) =f{y~ x ),y e G. We say that G is amenable if there exists an m <E L°°(G)* such 
that m > 0, m(l) = 1 and m(J) = m(f) for any x e G and / € L°°(G). Properties of 
amenable groups can be found in Greenleaf [13], Paterson [20] and Pier [21]. For any 
two measurable functions / and g on G, their convolution / * g is defined by 

/**(*)= fjmg(r x x)dt 

whenever this makes sense. The Figa-Talamanca-Herz algebra A P (G) is the space of func- 
tions u: G — > C which can be represented, nonuniquely, as 

oo oo 

u = £ v„ * u„ for u n e lf{G) and v„ G L<(G) with £ ||«„y vj, < oo 

and = inf ||w n || P ||v„|| 9 , where the infimum is taken over all possible represen- 
tations of u and J + ^ = 1 . It is known that A P {G) is a regular tauberian algebra un- 
der the pointwise multiplication and its Gelfand spectrum is G. Furthermore the algebra 
A P (G) C Co(G) and it has a bounded approximate identity if and only if G is amenable. 
Up = 2, A P (G) = A(G), the Fourier algebra of G. Each/ <E 0{G) defines a bounded 
linear functional on A P (G) by 

if, u) = J G f(x)u(x)dx for u 6 A P (G), 
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and it defines a bounded linear operator on LP{G) by f(g) = f * g,g G Lf(G). The norm 
of / as an element of A P (G)* and the operator norm off as an element of ^(ifiGj) are 
the same, that is, 

ll/lk(0= sup |</»| and 11/11 , v= sup 

«^p(G),||«II<1 V (Q J g€^(G).||g||,<1 

are equal. It follows that 0(G) is a subspace of /Lj(G)*. Let PF p (G) and PM P (G) be the 
closures of L'(G) in /Lp(G)* with respect to the norm topology and the weak* topology 
respectively. It can be shown that PM P (G) — A P (G)*. When p = 2, PM2(G) is usually 
denoted by VN(G) while PF 2 (G) is C* P (G), the reduced group C* -algebra of G. When G 
is amenable, C* p (G) = C*(G), the group C* -algebra of G. 

B P (G) is the pointwise multiplier algebra of A P (G), consisting of the continuous func- 
tions v on G such that uv G A P (G) for all u G A P (G). The norm on B P (G) is defined 
by 

\\v\\b p{ G) = sup{\\vu\\ Ap(G ) and \\u\\ Ap{G) < 1 }. 

Observe that A P (G) C fl p (G) and if v G /* P (G), HvH^o < ||v||^g). It has been showed 
that PF P (G)* = B P (G) if and only if G is amenable (see Cowling [5]), in this case, for 
any b G B p (G), b G PF P (G)* is defined by {bj) = S G b(t)f(t)dt for/ G L\G). When 
p = 2, 52(G) = fl(G), the Fourier-Stieltjes algebra of G which is the dual of the group 
C* -algebra C*(G). 

For any/ G PA/ / ,(G) and b G ^(G) or 6 G 5^(G),/ o 6 is a functional on PM P (G) or 
PF P (G) defined by (f o b,g) = (b,g •/) whenever this makes sense, where g • f is the 
product of g and / in PM p (G). Let x G G, 5 X denotes the point measure at *. 

Let X be a Banach space. We recall definitions of some geometric properties of X as 
follows. 

Radon-Nikodym property (RNP). The Banach space X is said to have the RNP if each 
closed convex subset D of X is dentable, i.e. for any e > 0 there exists an jc G D such 
that x $ Co(D ~ B f (x)), where B t (x) = {y € X : \\x -y\\ < e}. It is known that has 
the RNP if and only if every separable subspace of X has a separable dual (see Bourgin 
[3], Diestel [6] and Huff [ 1 6] for more information). Granirer [11] proved that A P (G) has 
the RNP if G is compact. 

Dunford-Pettis property (DPP). The Banach space X is said to have the DPP if, for any 
Banach space Y, every weakly compact linear operator u:X-^ Y sends weakly Cauchy 
sequences into norm convergent sequences. Sec Diestel [7] for more information on this 
property. 

Schur property. The Banach space X is said to have the Schur property if every weakly 
convergent sequence is norm convergent. 

Let A be an arbitrary Banach algebra. For any a G A and / G A*, af G A*[fa] is 
defined by (af, b) = (f,ba)[{fa,b) = (f,ab)]. The functional / is said to be weakly 
almost periodic [almost periodic] on A if {af : a G A,\\a\\ < 1} is relatively weakly 
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compact [relatively compact]. By wap(/l) [ap(/l)] we denote the linear subspace of A* 
consisting of all weakly almost periodic [almost periodic] functional on A. 

3. The DPP, the RNP and the Schur property of A P (G) and B P (G). In this section 
we will be concerned with geometric properties of A p (G) and B P (G). A number of suffi- 
cient conditions for G to be compact will be established. Our main tool is the following 
lemma. We give an elementary proof here. 

LEMMA 3.1. Let G be a locally compact group. If G is not compact then there is a 
sequence {x n } in G such that for any compact set KofG there is an N with x„ K for 
all n> N. Furthermore, for any a G A P (G), Xn a — » 0 weakly in A P {G). 

Proof. Since G is not compact we can choose an open and closed subgroup Go of 
G which is a-compact. Let G 0 = U£Li Kn, where K„ is an open set with compact closure 
and K n C K„+i for all n. For each n > 1, choose an x n G ~ K n . Then for any 
compact set K of G we have KDGo C [jJLi K > n K for some natural number N. So 
x n £ K for all n> N. 

Let a e A P (G). We now show that Xn a — * 0 weakly in A P (G). Let e > 0 and let 
F € PM p (G) be fixed. By the definition of A p {G) there are/„ G If(G) andg n G L q (G) 
such that a = E^i gn *fn and E£l| ||gn|| ? |^||p < oo. Since continuous functions with 
compact supports are dense in LP{G) and L q (G), there exists an u = £*= , v„ * u„ in A P (G) 
such that \\u — a\\ < e, where all u„ and v„ are continuous on G with compact supports. 
So \\ Xii u - Xn a\\ = \\u - a\\ < e for all n and 



To prove that F^a) — » 0, it suffices to show that (/^(v, * «,)) — » 0 for each 1 < i < k. 
Let /be fixed and let V be the support of v,. Then V is compact and |v,(f)| < ||v ( ||ooM0 for 
all /. Recall that if T is in PM P (G) and v * u is a simple element of A P (G) then ( T, v * u) = 
{T(u\ v). Hence we have the following 



\(F, X M * *t))\ = KF.GwV,) * Ui)\ (see Pier [21], pp. 14, (4)) 

= |(F( W ,-Uv,-)| 

< fj^UMm^Mdt 

«Nioo(A(F))'(^ lK i^)rA)'--o 




\(F, Xm a)\ <\(F, Xm a- Xl ,M)\ + \(F, Xm u)\ 

<IWIIL«-^II + I(^«>I 




Copyrighted material 



COMPACTNESS OF A LOCALLY COMPACT GROUP G 



mi 



as n — * oo since F(u,) G LP(G) and for any compact subset K of C7, x n 1 P is disjoint with 
A: for sufficiently large n, where q is a real number so that 1 jp + 1 jq = 1 . ■ 

Remark 3.2. This lemma can also be derived from Lemma 8.4 of [ 1 7] as follows. 
Lemma 8.4 of [17] implies that we have not only the inclusion A P (G) C vnp(PF p (G)) 
but also the inclusion A P (G) C vtap(PM p (G)). It follows that, for a G A P (G), the set 
{a x :x G G} is relatively weakly compact (q x = 8 X o a; see our Proposition 4.4 below). 
The set {8 x : x G G}, being the spectrum of the algebra A P (G), has the zero functional as 
the weak* cluster point if and only if G is not compact. Note that for any / G L l (G) and 
x G G, we have a x = (8 X o a,/) = ((/A)) * a(x), and (/A) * a G ^(G). It follows that 
if {x„} is any sequence in G that goes to "infinity", then a Xn — > 0 in the w*-topology. So 

— * 0 weakly in A P (G) (hence Xn a = (a x -i ) — ► 0 weakly; see Theorem 4.3). 

COROLLARY 3.3. Let G be a locally compact group. IfA p (G) has the Schur property 
then G is compact. 

PROOF. Suppose G is not compact. Let a G A P {G) be any nonzero element and let 
{x„} be a sequence as in Lemma 3.1. Then Xn a — ► 0 weakly in A P {G). But ||x„tf|| = 
||a|| 7A 0. So { Xn a} does not converge in norm. Therefore A P (G) does not have the Schur 
property. ■ 

COROLLARY 3.4. Let G be a locally compact group. For any nonzero element a G 
Ap(G), { x a : x G G} is a compact subset of A P {G) if and only if G is compact. 

Proof. Let G be compact. Let { x „a} be any net in the orbit { x a : x G G}. Since G 
is compact, we can assume that x a — ♦ x for some x £ G. Then Xa a — ► x a in A P {G) since 
translation is norm continuous (see Granirer and Leinert [12], pp. 466). Thus { x a : x G 
G} is compact. 

Conversely, suppose { x a : x G G} is compact. If G is not compact, then we can choose 
a sequence {x n } from G as in Lemma 3.1. It follows from Lemma 3.1 that for any a G 
A P (G), Xn a —* 0 weakly in A P (G). Since || XB a|| = ||a||, for any non-zero a G A P (G), { Xn a} 
does not have any subsequence which is convergent in norm. This is a contradiction. 
Thus G is compact. ■ 

Remark 3.5. This may fail if a G B P (G) ~ A p {G). For example, if G is a nonim- 
pact locally compact group then 1 G B P (G). So{ x l : x G G} = { 1 } is compact, although 
G is not compact. 

COROLLARY 3.6. Let G be an amenable locally compact group. If B p (G) has the 
RNP and the DPP then G is compact. 

Proof. Since B p (G) = PF P (G)* and B P {G) has the RNP and the DPP, it follows 
from Theorem 3 of [7] that B P (G) has the Schur property. Hence A P (G), the subspace of 
B p (G), also has the Schur property. Corollary 3.3 implies that G is compact. ■ 

Remark 3.7. Lau and Ulger [17] showed that B{G) has the RNP and the DPP if and 
only if G is compact. 

For any x G G and / G PM P (G), define J G PM P (G) by (/,») = (A,-.w) for all 
u G A P (G). It follows clearly that \\f\\ = ||/|| since \\ x - t u\\ = \\u\\ for all u G A P (G). 
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COROLLARY 3.8. Let Gbea locally compact group. If there is a sequence {x„ } in G 
as in Lemma 3. 1 then for any f G PM p (G), — > 0 in the w* -topology of PM p (G). Con- 
sequently, if there is a nonzero f G PM P (G) such that the right multiplication operator 
l f : PM P {G) -» PM P {G) defined by 7/(g) = g ffor g G PM p (G) is compact then G is a 
compact group. 

Proof. To prove the first claim, if {*„} in G is as in Lemma 3. 1 then it follows from 
Lemma 3.1 that for any u G A P (G), Uf,u) = (/",,„-. w) ~» 0. 

For the second claim, suppose G is noncompact, and let 7/ be a compact operator as 
defined above. Note that for any u G A P (G) and xGCwe have u8 x = x u G A p {G), since 
foTM\yheL l (G)CPF p (G), 

(ub x ,h) = (u,8 x • h) = (u, x -ih) = ( x u,h). 

It follows that7/&) = x ./since 

(7/(U") = {Sx /,«) = if,u6 x ) = {f, x u) = (,-,/">• 

Thus if Xm if -* 0 in w*-topology, then {lf{8 Xn )} has a subsequence {l/(S x J} which 
converges to 0 in norm by the compactness of the operator 7/. This contradicts || Xnl -i/|| = 
11/11 > 0 for all k. m 
That A P (G) has the RNP is close to A P (G) being a dual space (see Theorem 4. 1 of 
Taylor [22]). Recall that for any T G PM P (G), the support of T, denoted by supp(r), is 
a subset of G defined by x G supp(7') if and only if for all u G A P {G) we have u • T = 0 
implies u(x) = 0. Let £ be a subset of G. Let PM P (G)\ E denote the subset of PM P (G) 
consisting all the elements T G PM P {G) such that supp(r) is contained in E. 

PROPOSITION 3 .9. Let Gbea locally compact group and let G 0 be an open subgroup 
ofG. If A p (G) = X* for some Banach space X, then 

X\s a subspaceof PM P (G) and A P (G 0 ) = (X D PM p {G Q j)* . 



PROOF. Observe that X is a subspace of A"** = PM P (G) and A P (G 0 ) can be identified 
with the subalgebra of A P (G) consisting of functions which vanish outside G 0 (see Herz 
[14], Proposition 5). Thus, PM P (G 0 ) * PM p (G)\ Go and every element in A P (G 0 ) is a 
bounded linear functional on XDPM p (Go). Let u G (XC\ PM P (G Q ))\ Then by Hahn- 
Banach Theorem, u can be extended to an element u in X*. It follows that w G X* = 
A P {G). Since for any/ G X(l PM P (G 0 ), u(f) = u\c 0 <f), we have u = u\xnPM p (G 0 ) = ii\c 0 
and u is an element of A P (G 0 ) since u\ Go G A P (G 0 ) (see Herz [14], Proposition 5). ■ 

COROLLARY 3.10. Let Gbea metrizable locally compact group. IfA p (G) is a dual 
space then A P (G) has the RNP. 

PROOF. It suffices to show that any separable subspace of A P (G) has the RNP (see 
Huff [16], page 77). Since any separable subspace of A P (G) is contained in A P (G 0 ) for 
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some open cr-compact subgroup of G, we only show that A P (G 0 ) has the RNP. If A P (G) 
is a dual space then it follows from Proposition 3.9 that A p (Go) is a separable dual space. 
Hence A P (G 0 ) has the RNP. ■ 

Remark 3.11. Let G be a locally compact group. If p = 2 then A 2 (G) has the RNP 
implies that A 2 (G) is a dual space (see Taylor [22] Theorem 3.5). 

4. Almost periodic functionals on PF P (G). In this section we investigate when 
elements of A P (G) or B P (G) are almost periodic functionals on PF P (G). When £'(G) C 
PF P (G), we will see that it is easy to prove our main result Theorem 4.7 by applying 
Lemma 3.1. We like to know exactly for which groups that £'(G) C PF p (G) holds, that 
is, PF P (G) has an identity. The following is a consequence of results of Granirer [10]. 

Proposition 4.1. Let G be a locally compact group. t\G)C PF p (G) if and only if 
G is discrete. 

Proof. It follows from the definition that if G is discrete then i\G) = L\G) C 
PF P (G). 

Conversely, let £'(G) C PF P (G). Then8 e G PF P (G). Assume that G is separable first. 
So PF P (G) is separable. By the fact that 8 e is the identity of PF P (G) together with the 
Corollary of Granirer [10] on page 128, which says that PF P {G) C UC P (G) C C P (G) 
for any locally compact group G, we have that PF P {G) = UC P (G) = C P (G), where 
UC P (G) = A P (G) ■ PM P (G) and C P (G) = {T G PM P (G) : Tf +f 2 T G PF P (G) if/,,/ 2 G 
PF P (G)}. Theorem 16 of Granirer [10] implies that G is discrete if we take PF P {G) as X 
in that Theorem. 

Let G be an arbitrary locally compact group with l\G) C PF P {G). Take any open- 
closed compactly generated subgroup Go of G. Then l l (Go) C PF p {Gq). In fact, 
PM P (G 0 ) = PM P (G) | G 0 (see Herz [14], Proposition 5). So /^(Go) = PF P (G) \ G 0 . 
For any x € G 0 ,S X € PF P (G) implies^ G PF P (G 0 ). Therefore ^(Go) C PF P (G 0 ). If G 0 
is discrete we are done. Otherwise there is a compact normal subgroup K in Go such that 
X(K) = 0 and Gq/K is separable (see Theorem 8.7 of Hewitt and Ross [15]). Now we 
prove that l\G 0 /K) C PF P (G 0 /K). Let* G G 0 . So8 x G PF P (G 0 ). Let/, G L'(G 0 ) and 
W&x —fn\\ — * 0 in PM p (Gq). LetA p (Go)K be the subalgebra of /* p (Go) consisting of func- 
tions which are constant on cosets of K. Then A P (G 0 /K) ^ A P (G 0 ) K by Proposition 6 
of Herz [14]. If we consider 6, and/„ as elements of PM P (G 0 / K), \\f n - b x \\ -+ 0 since 
IIW-UwcwJ < lUi-ftrll- So« rf G FF P (G 0 //Oand £'(G 0 //0 C PF P (G 0 /K). Thus, 
G 0 is discrete. It follows that K is open. This is a contradiction since X(K) = 0. ■ 

Remark 4.2. (a) If A p {G)* = PF p (G) then G is discrete since £'(G) C /^(G)* and 
so £'(G) C PF^G) always holds. This was proved by Forrest [9]. 

(b) Let G be a discrete group. Then £'(G) C PF P (G). By Lemma 3.1, it is easy to 
prove that ap(PF p (G)) = fl p (G) implies that G is compact. In fact, if G is not compact, let 
{x„} in G be a sequence as in Lemma 3. 1 and a G A P {G) be such that a(e) = 1 . Then 8 Xh o 
a -* 0 weakly in A P (G) (see Proposition 4.4 below). But \\S Xm o a|| = |^| > a(e) ^ 0. 
So {6 X „ oa} does not have any subsequence converging in norm. Hence a g ap(PF p (G)). 
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THEOREM 4.3. Let Gbea locally compact group. For any real numbers p > 1 and 
q > 1 with 1 lp + 1 jq = 1, the operator T:A p (G) — > A q (G) defined by T(u) = u is 
an isometric isomorphism of Banach algebras A p (G) onto A q (G). Hence V: PM q (G) — » 
PM P (G) is a linear isometry from PM q (G) onto PM P (G). Furthermore, the restriction V R 
is a linear isometry from PF q (G) onto PF P (G). 

Proof. By definition, \\u\\ = inf Y%L\ \\i*n\\p\\v„\\ q , where the infimum is taken over 
all possible representations u = v„ * u n for u„ G LP(G) and v„ G L q (G) with 
££i IkMhll, < 00, Hence T(u) = u = u„ * v„ for u„ G W{G), v„ G L q (G) with 
E£i IkUM* < oo (see Pier [21], pp. 14 (3)). So G A q (G) and ||r( M )||^ (G) = 
II "IU P ((7) hy the definition. It is clear that T is an algebraic isomorphism and V is a linear 
isometry from PM q (G) onto PM P (G). 

Now we show V R is a linear isometry from PF q (G) onto PF P (G). Since L'(G) is dense 
in PF q (G) and r* is a linear isometry of Banach algebras, it is sufficient to show that 
r<f) G PF P (G) for all/ € 0(G) C PF^G). Let/ € L'(G) and « G /^(G), then 

So T(/) =/A is in L\G) C PF P (G). u 

PROPOSITION 4.4. Le/ G be a locally compact group. Then 8 x ou = u x for allx G G 
aw/ w G A p (G). Furthermore, if{x n } is as in Lemma 3. 1 and a G A p (G) is any element, 
then ^ oa-»0 weakly in A P (G). 

Proof. We show that<5 v ow = u x first. For all / G l'(G) C P/>(G), by the definition, 

<*,°«,/> = («,/*$,) = («,A(x-'K ,) = <«„/>. 

Thus, 8 x ou = u x since both and are elements of PF P (G)*. 

For any F p G PM p (G), by Theorem 4.3, there is an F G PM^G) such that T*(F) = F p . 
Hence, by applying Lemma 3. 1 to A q (G), we have 

(F p ,5 Xn oa) = <r(F),<0 = ■ 

To prove our main theorem for general G, we need the following lemma which is a 
nondiscrete version of Lemma 3.1. 

LEMMA 4.5. Let G be a locally compact group. If there is a sequence {x„} in G as 
in Lemma 3. 1 then h n o a G A P (G) and h„ o a — ► 0 weakly in A P (G), where a G A p (G) 
is any element and h„ = Xa 1 w G PF p (G) for any compact subset W of G with positive 
measure. 

Proof. For all/ e L\G) C PF p (G), we have 

(h n oa,f) = (a,f*h n ) 

= f G a(t)f(y)h„(y~ l t)dydt 

= f c a(t)f(y)h n (r l y)dtdy 
= {a * h m J), 
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Hence h„oa = a*h„. To show that h„oa G A p (G) y let a* G A P (G) be with compact support 
and ||a* — a\\ —> 0. Then h„ o a* G and ||A„ oa — h„o a*|| < ||/t„|| ||a* — a\\ — ► 0 

uniformly for n as — » oo. Hence h n oa € A P (G). To prove that A„ o a — > 0 weakly in 
/4p(G), we can assume that the support of a is compact. So a € L q {G), where q is a real 
number such that \/p + \/q = 1. For any F p G PM P (G), by Theorem 4.3, there is an 
F € PM q (G) such that = F p . Thus we have 



as n ► oo since F(a) G lf(G), and for any compact subset K of G, x~ l W and AT are 
disjoint for sufficiently large n. ■ 

REMARK 4.6. This lemma can also be derived from our Corollary 3.8 and Corol- 
lary 8.5 of Lau and Ulger in [17]. 

The following theorem improves some of the results of Lau and Ulger [1 7]. The equiv- 
alence (1) <=> (2) is a generalization of the equivalence (a) (J) in Theorem 4.5 of Lau 
and Ulger [17]. It also improves Corollary 8.7 of Lau and Ulger [17]. Each of the equiv- 
alences (1) (3) and (1) (4) improves (a) & (b) in Theorem 8.8 of Lau and Ulger 
[17]. The equivalence (1) (5) is in Theorem 8.8 of Lau and Ulger [17]; here we give 
a different proof for (5) =*(!). 

THEOREM 4.7. Let Gbea locally compact group. Then the following are equivalent 

(1) G is compact. 

(2) ap(PF p (G)) = B P (G) 

(3) A P (G) C ap(FF p (G)) 

(4) A p (G)n & p{PF p (G)) ^ {0} 

(5) A P (G) C *p{PM p (G)). 

PROOF. (1) (2) In Corollary 8.7 of Lau and Ulger [17], it is proved that compact- 
ness of G implies ap(PF p (G)) = B P (G). 

(2) => (3) (4) It is clear since A P (G) C B P (G) and A P (G) ? {0}. 

(4) => (1) Let G be noncompact and let a G A P {G) PI ap(PF p (G)) be nonzero. Let 
W be a compact subset of G such that \ w * a(e) = J c \ w {t)a{t)dt ^ 0 and let h„ be 
denned as in Lemma 4.5. Then h„ o a G A P {G) and h„ o a — > 0 weakly in A P (G). Since 
a G /tp(G)Dap(PF p ((7)), we assume that \\h„ oa\\ — » 0 in the norm topology of A P (G) by 
passing to a subsequence. As h„oa = a *h n (see the proof of Lemma 4. 5) and 6 X G /1 P (G)* 



\(F p ,h n oa)\ = \(nF) t a*h n )\ 

= \(F,h„*a)\ (see Pier [21], pp. 14,(4)) 
= \(F(a), x M)\ 
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has norm 1 for any x e G, wehave ||/j„oa|| >6 Xn (a*h„) — (h n *a%x„~ x ) = \ w *a(e) > 0 
for all n. This is a contradiction. 

(1) => (5) It was proved in Theorem 8.8 of Lau and Ulger [17]. 

(5) => ( 1 ) If G is not compact, we choose {x„} from G as in Lemma 3. 1 and a € A P (G) 
with a(e) > 0. It follows from Proposition 4.4 that b Xn o a = a Xn and 8 Xn oa—>0 weakly 
in A p (G). Thus, 6 Xii o a — » 0 in the w* -topology of PM P (G)* as well. By (5), we assume 
that 6 Xn o a — ► 0 in norm by passing to a subsequence. But \\6 Xil o a\\ = ||a x J| > a(e) > 0 
for all n. This is impossible. Therefore, G is compact. ■ 

Remark 4.8. It is interesting to compare (1) <=► (3) and (1) <=► (4) of our The- 
orem 4.6 with Corollary 8.5 of Lau and Ulger [17], where it has been showed that 
A P (G) C wap(f > F p (G)) for all locally compact groups G. 

5. Compactness of G and some operators. For any Banach algebra/*, Arens [1], 
[2] introduced two natural Banach algebra products on A**, each of which is an exten- 
sion of the original product in A when A is canonically embedded in A**. Duncan and 
Hosseiniun showed in Lemma 3 in [8] that A is an ideal under either Arens product if and 
only if all the multiplication operators on PF P (G) are weakly compact (see also Ulger 
[23] and Forrest [9]). Brian Forrest [9] proved that A P (G) is an ideal in its second dual 
if and only if G is discrete. The main result of this section is Theorem 5.2, which shows 
that PF P (G) is an ideal in its second dual if and only if G is compact. It also generalizes 
some results of Lau and Ulger [17]. For its proof we need the following lemma. 

Lemma 5.1. Let G be a locally compact group. If{x„ }isa sequence as in Lemma 3. 1, 
then Tfaly) -* 0 and T f ((\y) Xii &(x n )) — * 0 in the w* -topology of PM p (G), where 
Tfi PF P (G) -» PF P (G) is defined by Tf{h) =f hforallh£ PF P (G), V is any com- 
pact subset of G, andf € L l (G). 

PROOF. Let u € A P (G). By applying Corollary 3.8, we have 

(t/G, 1 y), u) = { Xn 1 y, (A/) * u) - 0 as (Aft * u <E A p {G). 

Note that (r/((l k)x. «) = (f* Ir.tt^i). By Theorem 4.3, let/* \ v = r R {h) for 
some h e PF q {G). Then (t^ 1 y), u) = ) = (*. x„u) 0 by Lemma 3.1. ■ 

In the following theorem, the equivalence (1) & (3) generalizes Theorem 6.7 (a) of 
Lau and Ulger [17], and the equivalence (1) (5) is a parallel result of Forrest [9]. 

THEOREM 5.2. Let Gbea locally compact group. Then the following are equivalent: 

(1) G is compact. 

(2) Tf PF P (G) -> PF P (G) is compact for allf G PF P (G), where r f (g) =f-g. 

Furthermore, if G is amenable, then all the conditions from (1) to (5) are equivalent. 

(3) T f : PF P {G) PF P (G) is weakly compact for all / e P/>((7), where T f (g) =f g. 

(4) if. PF P {G) PF P (G) is weakly compact for all/ € PF P (G), where 7/(g) = g f. 

(5) PF P {G) is an ideal in its second dual under either Arens product. 
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PROOF. (1 ) =» (2) This follows from Lemma 8.6 of Lau and Olger [ 1 7]. 

(2) =$► ( 1 ) and (3) => ( 1 ) Let G be noncompact. Then we can have a sequence {x„ } in 
G as in Lemma 5.1 (see Lemma 3. 1 ). Let V and V be compact subsets of G with positive 
measures and / = 1 [/. 

Let (2) holds. As rf PF P {G) — > PF P (G) is compact and 

IKirMMI < IOfMMIi - MV) 

for all n, we may assume that y ) Xii A(x„)) — » 0 in the norm topology of PF P (G) by 
passing to a subsequence since, by Lemma 5.1, it converges to 0 in the w* -topology of 
A P (G)\ Hence we get 

\\r f ((lv) x Mx»))\\ = \\f* ((U) x Mx n ))\\ 

= \\f*(\y* VOII (see Pier [21], pp. 14(2)) 

>ll/"*(ir*V')*M 

= |j/-*l K ||>Oforalln. 

This is a contradiction. 

Assume (3) is true. By a similar argument as above, we can assume that rfa 1 v) — > 0 
in the weak topology of PF P {G). Then since 1 G B p {G) = PF P (G)\ 

(LTfU. i k)> - jf/W* i rXr l y) dt dy 

= f G lu(t)(*Jv)(r l y)dydt 
= \(U)\(V) > 0. 

This is a contradiction. 

(1) => (3) is trivial since (1) (2). 

(1) =>• (4) follows from Lemma 8.6 of Lau and Ulger [17]. 

(4) => (1) Let r R :PF p (G) — » PF q (G) be the linear isometry as in Theorem 4.3. We 
prove that r R (f) ■ g = 7^((Ag) •/) for any/ € PF P (G) and g G PF q (G). Assume that 
f,g G 0{G) and m G A q (G). Then, since go U = «*g (see the proof of Lemma 4.5), 

(r R (o-g,u) = (r R (f),gou) 

= {f,Tg(g on)) 

=<r,7jK«**)) 

That{g-/: ||g|| < \, g ePF p (G)} is weakly compact implies that {r R (f) g : \\g\\ < 1, 
| g G PF q (G)} is weakly compact. So G is compact by (3) ( 1 ). 

(1) (5) follows from (3), (4) and Lemma 3 of Duncan and Hosseiniun [8] 
(p. 318). ■ 
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COROLLARY 5.3. Let G be a locally compact group. If every bounded linear operator 
T: PF p {G) —* B P (G) is compact then G is compact. Conversely, ifG is compact then every 
bounded linear operator T: PF P (G) — » B P (G) is weakly compact. 

PROOF. Suppose that every bounded linear operator T: PF P (G) — » B P (G) is compact 
and G is not compact. Let h„ G PF P (G) and a G A P (G) be as in Lemma 4.5. Define 
T:PF p (G) -> B P (G) by Tif) = / o a for all / € PF P (G). Then T is a bounded linear 
operator. So it is a compact operator. Thus, we assume h„ o a — ► u in norm for some 
u G B P (G) by passing to a subsequence. By Lemma 4.5, h n o a — > 0 weakly in ^ P (G). 
Since IMI^g) < ||«|| if " G /lp((7), A„oa — * 0 weakly in B P {G) as well. So h„oa — > 0 
in the norm of B P (G). But ||/j„ o a|| > 6 X(t (a * h"„) = (/i„ * aXJcT 1 ) = 1^ * a(e) > 0 for 
all n if we take W and a properly. This is a contradiction. 

Conversely, let G be compact. Suppose T: PF P (G) — » fi p (G) is any bounded linear 
operator. To show that 7T is weakly compact, let f„ G PF P {G) with |[/^|| < 1. Since G 
is compact, fl p (G) has the RNP by Theorem 2 of Granirer [1 1]. So the predual PF p (G) 
has no copy of i x . By applying Rosenthal's -theorem, we can assume that {f n } is a 
weak Cauchy sequence by passing to a subsequence. Next, we show that {T(f„)} is a 
weak Cauchy sequence. Let a G B P (G)*. Define d G PF P (G)* by d(/) = (a, T(f)). 
Then d is bounded since both T and or are bounded. So {dt(f„)} is a Cauchy sequence, 
/.e. {(a, T{f n ))} is a Cauchy sequence. Since G is compact, ^ P (G) = B P (G) is weakly 
sequentially complete by Lemma 18 of Granirer [10]. Thus there is an b G B P (G) such 
that T(f n ) -i ft weakly in B P (G). m 

COROLLARY 5.4. Let G be a discrete group. Then every bounded linear operator 
T: PF P (G) — ► B P (G) is compact if and only ifG is finite. 

Remark 5.5. For discrete groups, this generalizes the equivalence (a) & (h) in 
Theorem 4.5 of Lau and Ulger [ 1 7], where this result for p = 2 was proved by using 
C* -algebraic techniques. 
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SUFFICIENTLY HOMOGENEOUS CLOSED 
EMBEDDINGS OF /V -1 INTO A" ARE LINEAR 

PETER RUSSELL 



Abstract. We show that over a field k of characteristic zero an a time (n — I)- 
space AjJ -1 embedded as a closed subvariety in affine n-space AJ and homogeneous for 
a codimension two linear torus action on AjJ is defined by the vanishing of a variable. 

To determine the nature of closed embeddings of affine /w-space /V into affine n- 
space A" is a central problem in the geometry of affine spaces. The famous "epimorphism 
theorem" of Abhyankar-Moh [AM] and Suzuki [S] settles the case m = 1 , n = 2 over a 
field k of characteristic zero: A "line" Aj embedded in the "plane" Aj[ is defined by the 
vanishing of a variable. (A variable on A" is a function y = y\ that extends to a global 
coordinate system yi,y2, •••,yn for A".) In most other cases, only very partial results 
have been obtained so far, even when, or better particularly when m = n — 1 (see [RS], 
for instance). With the epimorphism theorem as a starting point we will show that if 
char* = 0 and if Y ~ AJp 1 is closed in X ~ AJ[ and sufficiently homogeneous in the 
sense that it is invariant under a (n - 2)-torus T ~ G^ 2 acting linearly and effectively 
on X, then again Y is defined by a variable. (Special cases of this result with n = 3 were 
considered in [PTD]). Our proof, an induction taking off from n = 2 and the epimorphism 
theorem, quite naturally forces us to consider certain finite group actions in addition to 
those of tori. We show: 

MAIN THEOREM. Let kbea field of characteristic zero and suppose 

AT 1 ^YcX~W k 
with Y closed in X and invariant under an effective linear action of 

G=TxF 

where T ~ G r mJ[ is a torus of dimension r > n — 2 and F a finite abelian group. Then 
there exists a system of G-semi-invariant variables y\,. . . ,y n for X such that Y is the zero 
locus ofy\. 

In a sense this is only a modest contribution to the embedding problem since it begs 
the main question for our homogeneous situation, namely: "Is every effective action of a 
(n—2)-torus on affine n-space UnearizableT We were motivated to undertake the present 
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study largely to gain some understanding of the difficulties one is likely to encounter 
trying to resolve this question by an inductive procedure along the lines of [KR 1 J, starting 
with the expected positive answer for n = 3 (see [KR3], [KR4]). 

1 . Notation and preliminary results. We fix a field k of characteristic zero and set 

A = k [n] (the polynomial ring in n variables over*), 
X= h" k = Spec/f, 
T = G r mJi the /--dimensional torus over k). 

Let F be a finite abelian group and suppose 

G~TxF 

acts linearly and effectively on X. We call n - r the codimension of the action. Let 

G~ TxF 

be the character group of G. We note that T is canonically determined as a subgroup of 
G and t as a quotient of G and that r = rank 0. The action of G can be diagonalized, 
that is 

(1.1) A = k[x u ...,x„] 
such that 

with Xt eG. Moreover, xi » • • • * X» generate & (since the action is effective). 

Hence, with respect to a fixed diagonalization, the action of G is given by a surjective 
homomorphism 

(1.2) tpiY-*G 

a = (a x ,...,a n )^x a = X"\ --'X a n 

and the action of G is in fact determined by 

£=Ker^CZ". 

1 .3. Definition - Lemma. Let 

(We use the shorthand notation a = (a, , . . . , a n ) and x" = jtj ■ • • *£•)• Then/ is a semi- 
invariant for G if and only if 

Supp/={a|/ a ^0} 
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is contained in a coset 

a*+E 

off. We call x = </>(<**) the weight off. 

1.4. Suppose x« is of infinite order in G. The stabilizer G' of (0, ... ,0, 1) then acts 
on A" = jc ~ 1 ( 1 ) — and there is an essentially bijective correspondence between 
G-homogeneous functions on X and G'-homogeneous functions on X which we now 
describe. 

Let p: 2" — » Z"~\ p{a\ ,...,a„) = (a\ , . . . , a„-\ ) be the projection on the first n — 1 
factors. We obtain a commutative diagram with exact rows 

0 — > E — > Z" G — >0 

4 '1 1 

0 — E> — ♦ Z-' X G/( Xn ) — >0 
where, by definition, 

<p'(a, , . . . , a„-! ) = x? • • • xjlj (mod x„) 

and 

E' = Kerip'. 

Since x« has infinite order, a„ is uniquely determined by (a\,...,a„-\) for any 
(ai,...,a„) € £. Hence 

p:E— >E? 

is an isomorphism and there is a unique homomorphism 

u: E' — ♦ Z 

such that 

p '(ai.---.0n-i) = a„-i,-«(ai,...,a„ i)) 

for (a u .. ,a„-\) G £'. 

G/(x») has rank r - 1 and hence is the character group of G' = T x F, where T is a 
(r - 1 )-torus and f is finite abelian. It is clear that G' is the stabilizer of (0, . . . , 0, 1 ) € X 
and that <p' t or E', describes the action of G' on A" = x~ l (l) in the sense of 1.2. Moreover, 
if 

H(x\ ,x„-\,x„) G k[x\, . . . ,x„-\ y x ny x n '] 

is G-homogeneous of weight \ E G, then H(x\,. . . ,x„-\, 1) is G'-homogeneous of 
weight \ = x(mod x«) € G'. Conversely, let 

h(x\,...,x„-\) = Yl^H^ e k[x\,... ,x„.\] 
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be G'-homogeneous of weight x € G'. Pick a* = (a*, . . . ,<_,) such that Supp/i C 
a* + £'.Then 

H(xi , . . . ,x„- 1 , x n ) = Y, hpxPx'* 0 -*) G k[x { , . . . , x„_ , , x„ , x~ 1 ] 
is G-homogeneous of weight <p(a\, . . . , a*_, , 0). 
1 .5. Remark, (i) We have on^xG 

H(x u - ■ ■ Xn) = Xi" 1 ' " " X«-"f' *0tl*l» • • • » X«-i^»-i) 



and H is uniquely determined by this relation. 

(ii) E is a submodule of finite index in E, the kernel of Z" — ♦ G — ► T ~ Z r . Let 7, be 
the image of \i in Z r . Then for = (Z>i, . . . ,b n -\) e Supp/i the unique corresponding 
term in H is 

hp?** with - a„7„ = ft - <)7i + • • - 4. t yy»-i. 

(iii) We have established a bijective correspondence between G-homogeneous ele- 
ments //of k[x\ ,x„-\,x„,x~ l ], up to multiplication by a power of x„, on the one hand 
and G'-homogeneous elements h of k[x\ , . . . , jc„_i ] on the other. 

(iv) In the passage from h to H there is a (generally non unique) choice of a* that 
will lead to H £ k[x\ , . . . , x„- \ , x„] and not divisible by x n , and the resulting H is unique. 

Let the notation be as in 1.1 and 1.5. In general, for Pi, . . . ,0 S G Z r , we denote by 
(01 ,.-,&)+ the subsemigroup they generate. We write T for the semigroup of the action 
of7-on*andnote([KbR]) 

r = (7i,...,7„) + . 

We recall that the following two conditions are equivalent ([KbR], [BH]). 

1.6. (i) r is unmixed, that is 

o^7er=>-7£r, 

(ii) the action of Ton X is fixpoin ted, that is, every T-orbit has a fixpoint in its closure. 

1.7. Definition - Lemma. Let the notation be as in 1.1 and 1.5. We denote by M the 
set of nonconstant T-invariant monomials in jci, . . . , x n and let 

Tr.X->X//T=SpecA T 

be the canonical map. 

(i) A T = k[M]. 

(ii) The nullcone (at 0) is defined to be 

= 1T 1 (*(<»). 

Xq is the union of all T-orbits with 0 in their closure, and the ideal of X Q in A is 
generated by M. Any irreducible component of^b is a linear subspace of A' with 
unmixed ^-action. 
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(iii) An irreducible x G A is called a nullvariable if x~'(0) C Xo or, equivalently, 
if x divides every element of M. In particular, x is a variable and in fact is part 
of any ^-homogeneous system of variables for X. Moreover, a nullvariable is 
(/-homogeneous. 

(iv) Let Y C X be closed and ^-invariant. Then Yf / T is naturally identified with a 
closed subset ofX/ / T and Y 0 = X 0 n Y. 

Proof. Only the G-homogeneity of a nullvariable needs commenting upon. It follows 
from: (a) The finite group F permutes the codimension 1 components of Xq. (b) Two 
nonproportional nul [variables have different T-weights. If not, one could replace the first 
by the second wherever it occurs in an invariant monomial and thereby construct an 
invariant monomial not divisible by the first. 

1.8. Remark. Suppose Tacts with codimension < 2 and let Y ~ AJJ - ' be closed in A' 
with Y (£ xj 1 (0), /' = 1 , . . . , n. Then T acts effectively on Y with codimension < 1 , and 
hence this action is linearizable by [BB]. In particular, the action has a fixpoint, which 
we may assume to be the origin of X. 

Our next two statements are certainly well known to the specialists. We include them 
for lack of proper references. 

1 .9. Proposition ("Equivariant Epimorphism Theorem"). Let the group G act 
linearly on k[x,y] and suppose f G k[x,y] is G-semi-invariant and defines a line (i.e. 
k[x,y]/f » Then there exists a G-semi-invariant g G k[x,y] such that k[x,y] = 
kV,gl 

Proof. The proof of the epimorphism theorem given in [A] shows: We may assume / 
is monic in y and degyf = deg/. Then there is a divisor d of deg^/ such that k[x, y] = 
k\f,g], where g is the ^/-approximate root w.r.t. y of /. By the uniqueness properties of 
approximate roots, g is G-semi-invariant. 

1.10. Lemma. Let B be a factorial affine k-domain and K its field of quotients. Let 
A be a B-algebra and u\,...,u r €A such that 

K®bA = K[u u ...,u r ]. 

Suppose that u\,...,u r are algebraically independent mod mA for every maximal ideal 
m ofB. Then 

A = fl[«i,...,u r ]. 

Proof. For each m, 

(B/m)[ui,...,u r ]^A/mA 

is injective by assumption, that is, mA Pi B[u x , . . . , u r ] = mB[u u . . . , u r ]. Given x G A, 
there exists p € B such that px G B[u\,. . . , u r ]. Hence if m 3 p, px G B[u\ , . . . , u r ] n 
pA C B[ui,...,u r ]nmA = mB[u u ... It follows thatpx G fl mB[u u ...,u r ] = 

pB[u ur], where p is the product of the distinct prime divisors of p. If we choose p 
with a minimal number of prime divisors, we have p-l. 
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2. The cases dim *//r=0 and r + dim* 7 > n- 1. We give part of the proof of 
the main theorem in this section. We note that 

dim* r <dimJi7/7 , <«-r<2. 

We keep the notation of 1 . 1 and 1 .7. We will assume tacitly that 0 € Y and that none of 
jci , . . . , x„ vanishes identically on Y. 

2.1. If dim*//r = 0, then the action of T on * is unmixed and no T-weight 7/ 
is 0. Hence if x\, say, appears as a monomial in the equation H of Y, we have, up to 
multiplication by a constant, H = x\ + H with H £ k[x 2 , . . . ,x n ] and G-homogeneous. 

2.2. LEMMA, (i) LetYcX be an irreducible, G-stable closed hypersurface such 
that Y T = X T and Y is smooth along X 7 . If 

r + dim* r >n- 1, 

then the defining equation for Y is part of a G-homogeneous system of variables forX. 
(ii) Suppose Y ~ A" -1 , r=n — 2 and dim Y 7 > I. Then the defining equation for Y 
is part of a G- homogeneous system of variables forX. 

Proof. In [KR1], Lemma 1.1 and Lemma 1.2, T-homogeneous variables defining Y 
are constructed, and it remains to verify only that they are part of a G-homogeneous 
system of variables. As to (i), we observe that iff = oa^\ - (3x r is the equation for Y 
as in the proof of [KR 1 ], Lemma 1.1, then a and 0 are G-homogeneous (and coprime), 
and we can choose a and b G-homogenous such that aa + b(3 = 1 . Then g = bxr+i + ax r 
is G-homogeneous as well. As to (ii), we note that in the case (i) of the proof of [KR1], 
Lemma 1 .2, A 1 ~ Y T C X 7 ~ A 2 is G-stable for the natural action of G on X T and hence, 
by 1 .9, defined by a variable that is part of a G-homogeneous system of variables for X 7 . 

2.3. By 2.2, the main theorem is proven in the following cases: 
0) r>«— 1, 

(ii) dimX 7 = 2, 
(Hi) dimX T = dim Y T = 1. 

2.4. Proof of the main theorem in case dim X 7 = 1 and dim Y 7 = 0. There is a unique 
variable, x\ say, with T- weight 0, and we have a G-equivariant decomposition A' = X T x V 
with X 7 = Spec k[x\ ],V= Spec k[x 2 , . . . ,*„]. 

(a) Suppose dim*/ / T = 1 . Then X 7 = X/ / T and Y = 0 x V. 

iff) Suppose dim*/ /T = 2. Then*// T = X 7 x Vj / 7 with V/ /T = SpecJt[ro], 
where m = x^ 2 ■ ■ ■ is ^-invariant. Now dim Y / /T < 1 since T acts effectively with 
codimension 1 on Y, and dim Yj /T = 0 is not possible since otherwise Y C Xo, which 
is of dimension < n — 2. Hence Y/ /T ~ A 1 . Moreover, since Y is smooth and Y 7 is 
connected, Y/ /T and X 7 meet normally in one point in X/ /T = Spec k[x\ , m], with 
m = 0 as equation for X 7 . Hence the equation for Yj / T in */ / T is of the form jc, + <f>(m) 
with <f>{m) e k[m], and the equation for Y in * is x, + <H*? ■ ■ ■ K'l so clearly part of 



Copyrighted mater 



1292 PETER RUSSELL 

3. The main induction. We continue the proof of the main theorem. By the results 
of Section 2 it remains to consider the case 

dim*//r>0and dim* 7 " = 0. 

Note that now X T = {0} and 0 E Y. We keep in force the assumption Y (f. xr'(0), 
i = 1 , . . . , n. 

3.1. LEMMA. Suppose x„isa nullvariable on X and 

H = x n + H 

is irreducible and G-homogeneous with H(0) = 0 and (0,...,0, 1) £ Supp/f. Then 

Proof. Since x„ is a nullvariable, there exists 

(*) or = (a,,...,a„) G £, ai,...,a n -\ > 0, a„ > 0. 

On the other hand, if H ^ 0, H has a monomial not divisible by x„. This gives a relation 
Xm = X? * " X„-i an d hence an element 

(**) 7 = (ci,...,c„_i,-l) G£, c,,...,c„_, >0. 

Since x„ is a nullvariable, the action of T on {jc„ = 0} is unmixed (see 1 .7 (ii)), that is, 
the semi group f = (7i, . . . ,l„-\)+ is unmixed. But (*) and (**) give -a„l„ € T' and 
fliiTn € T\ and this is not possible. So H = 0. 

3.2. Lemma. Ifx e Aisa nullvariable for X, then y = x\Yisa nullvariable for Y. 

Proof. By 1. 7(iii), x is one of the Say x = x n . By assumption, Y <f_X n = jc~'(0)and 
hence Z = X n C\Y consists of codimension one components of the nullcone Yq of Y, all 
passing through the origin (see 1 .7(iv)). If L is the linear part of the equation H of Y, then 
L is linearly independent of x n by 3.1, that is, Y and X„ meet normally in the origin in 
one irreducible component. It follows that v is irreducible on Y and hence a nullvariable 
by 1.7(iii). 

3.3. LEMMA. There exists xeA.xa nullvariable for X. 

Proof, (i) Suppose dim*// 7/ = 1. Then*// 7/ = k[m], where m is an invariant 
monomial, and any jc, dividing m is a nullvariable onX. 

(ii) Suppose dimX//T = 2. Then E C Z", the module of relations among the G- 
characters of the action (see 1 .2) is of rank 2 and contains two linearly independent ele- 
ments a = (au • • . »«■) and (3 = (b\ , . . . , b n ) with non-negative coefficients, i.e. 

a,/3 <E E+ =£n(z + y. 

For 1 = (ci,...,c) e Z", put supp^ = {i \ c, ^ 0}. Choosey € supp/3 such that 
Oy/fty = am{ai/bt}. Then a' = tya - aft G £+,a' and /3 are linearly independent, 



Copyrighted material 



SUFFICIENTLY HOMOGENEOUS CLOSED EMBEDDINGS 



1293 



and j g suppa'. So we may assume to begin with that supp/3 suppa and, repeating 
the argument if necessary, that supper £ supp/3 as well. Then every element of £+ is a 
linear combination with non-negative rational coefficients of a and 0 and hence x, is a 
nullvariable for X if and only if / e supp a Pi supp 0. 

So assume supp or n supp ft = 0. We may assume that the equation of Y is of the form 

H = x„+H 

where H ^ 0 has a monomial not divisible by x n . As in the Proof of 3. 1 we obtain an 
clement 

7 = (c,,...,c„_i,-l) € E 

with c, > 0, z = !,...,» — 1. We claim that a, 0,1 are linearly independent, in con- 
tradiction to rank E = 2. In fact, this is obvious if n & suppa U supp/?. So suppose 
n € supp a, say. Now supp a has at least two elements (since X 7 = {0}) and we may 
assume n — 1 € suppa as well, that is, a = (a t , . . . ,a„_i,a„) with a„_i > 0, a„ > 0. A 
nontrivial relation act + b(3 + cry = 0 is then not compatible with supp a D supp/? = 0. 

3.1. The induction. Suppose now n > 3. In view of 3.2 and 3.3 we may assume that 
x„, say, is a nullvariable on X and Y and that the linear part of the equation H for Y does 
not depend on x n . Put X 1 = x~ 1 ( 1 ) and T = X D y. We are then in the situation of 1 .4. 
By 1 .8, the action of G on Y is linearizable and by 1 .7(iii), x n restricts to a variable on K. 
Hence f ~ A" -2 . By induction on n (for « = 2 we invoke the equivariant epimorphism 
theorem) we may assume that 

h(x\,...,x„-\) = H(x\,...,x„-\, 1) 

is part of a G'-homogeneous system of variables 

h\ = h,h2, ...,An-l 

for X . Sayxi appears in the linear part of//. Free to modify jci byjci-free terms in H, we 
may assume to begin with that no term in H is of the form jc/^, c > 0, / > 2. Also, a term 
> 1, cannot appear since otherwise l„ = 0. Hence h\ = x\ +h\ withordo^i > 1. 
After a G'-homogeneous linear change of variables we may then assume that A, = x, + A, 
with ordo h, > 1 for i = 2, . . . ,n - 1 as well. 

Now pick a* = (0, . . . , 1 , . . . , 0) (a 1 in place /') and let //, be defined by hi and a* as 

in 1 .4. Then //,=// (see 1 .5(iv)). Let i > 2 and consider 0 = (b , 6„_, ) G Supp h<. 

By 1 .5(ii), the corresponding monomial in //, is with 

(*) -U = cm7i + • • • +(A - 1)7. + • • • + ViVi- 

Since (7i, . . . ,7„_i)+ = T' is unmixed and 



(**) 
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(see the Proof of 3.1), b„ < 0 can occur in (*) for b t = 0 only. Suppose it does, for 
i — n — 1, say. We then have 

e-7,,-, + r\ 

and in view of (**) this can occur for finitely many b„ < 0 only. Fix one of them. 
We then have "relations" 

£\ = {b\,...,b„-2, — \,b n ) G E, 

£2 = (C],...,C„-2,C n -i,C„) e E, 

with bu.. .,b„-2 > 0, b„ < 0, ci, . . . ,c„-\ > 0 and c„ > 0. Suppose we have e\ = 
(b\,...,b' a _ 2 , — l,b„) G E as well with b\,..., b'„_ 2 > 0 and e\ ^ ej.Thene', is not 
in the Q-span of t| and £2, and since rank E = 2, ei and £2 are linearly dependent and 
hence b\ = ■■• = b n - 2 = 0, -7„_i = 0 and 6,Ti +• • • +6^,_ 2 7 w -2 = 0 with b\ > 0 

for at least one i. We therefore have an invariant monomial X* • • x n "z\ not divisible by 
jc„, and jc„ is not a nullvariable. 

Hence for each / = 2, . . . , n — 1 there are only finitely many = (b\ t .. .,b„-\) € 
(Z+)" -1 such that (*) holds with b„ < 0. Moreover, 6, = 0 in that case. We can therefore 
modify A2 by a G'-homogeneous polynomial in h\,hi . . . ,h„-\ to make sure this case 
does not occur, then do the same successively for hy, . . . ,h n -\. We will then have for 
i fas {,..., n- 1 

(* * *) Hi = Xi + Hi E k\x\,.. . , ,x„] 

with ordo a§ > 1 . 

Now in k[G][x\ x n -\ ], the homomorphism ij: x t i—> //, is just conjugation by the 

automorphism jc, >— ► x^/» and it is defined over k[x n ,x~ l ] C with jc„ = \„. Hence 

. . . t x n -\,x„,x n ] = k[H\, . . . ,H„-\,x„,x n J. 

Moreover, by (***), H\,...,H n -\ are algebraically independent modjt„. By 1 . 10, applied 
to B = k[x„] C A, we have 

This finishes the induction. 
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Abstract. Our main result implies that for any choice 1 < m < n < p of integers 
there exist finitary algebras A\ and A 2 that generate the same variety, and such that 
the initial ^-segments of their centralizer clones coincide exactly when k < m, are 
isomorphic exactly when k < n and are elementarily equivalent exactly when k < p. 
The proof uses the existence and properties of disciplined topological spaces which we 
introduce and investigate here. 

1. Introduction. 

1.1. In simplest terms, the centralizer clone C\o(A) of a universal algebra A is the cat- 
egory of all homomorphisms between finite powers A 0 = {0}, A 1 = A, A 2 , A 3 , ... of 
A. For any integer r > 1, all homomorphisms between the powers A° y A\...A r form 
what is called the initial r-segment C\o r (A) of Clo(A). It is clear that the equality of cen- 
tralizer clone segments C\o r (A\) and C\o r {A{) of algebras A\, A2 always implies their 
isomorphism, and that their isomorphism always implies their elementary equivalence. 
The present paper demonstrates that, subject only to these obvious dependencies, the 
three upper boundaries of equality, isomorphism and elementary equivalence of central- 
izer clone segments of universal algebras may be chosen at will. 

This result translates an earlier one [7] on continuous maps of metrizable spaces, 
and exploits the method used by the first author in her preprint [10] to separate equality 
from elementary equivalence for centralizer clone segments of universal algebras. These 
two sources, in rum, derive their essence from yet another first author's paper [9] on 
continuous maps of metrizable topological spaces. The original impetus for [10] came 
from Ralph McKenzie who suggested that, having completed [9], the first author also 
investigate centralizer clones of universal algebras. 

Results of [7] on continuous maps of metrizable spaces can be 'translated' into our 
present results on homomorphisms of universal algebras only for metrizable spaces 
which are disciplined (see 2.2 below). We show that metrizable spaces constructed in 
[7] are disciplined and have other properties enabling the translation to proceed. 
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A somewhat more general notion of a clone is needed for an accurate description of 
our results. 

1.2. Let u) = {0, 1 , 2, . . .} denote the set consisting of all finite ordinals n = {0, . . . , 
*-l}. 

A subcategory T of the category Set of all sets and maps is called a clone on a non- 
empty set X if the set obj T of all objects of T consists of all finite Cartesian powers 
X° = {0}, X x = X, X 1 = X x X, . . . of X, the product projections p[ m) : A 7 " -» X with 
i G m are morphisms in T for any m G u\ and every A 7 " G obj T accompanied by all 
Cartesian product projections p^: X m — > X with /' G m is a categorical product, in T, of 
m copies of X This means that for any m,n G uj and any m-tuple/: A" — > A' of maps in 
7\ the map/: A 71 —►A 7 " defined by 

/(y) = (/oCv),...,/ w -.(y))foraii^Gr 

also belongs to T. In particular, for each rp:m—>n with m,n £ lj, the category T contains 
the map p w :X" — ► A 7 " given for every ^ = (yo, . . . ,y n -\) G A 7 * by 

p W (yo, ■ ■ • , V„-| ) = (y^ 0 ), • • • ,^m-l))- 

Set rr(m) = A 7 " for any m € u, and Tr(V') = P 1 ^ 1 for any mapping xj;:m —* n with 
m,n G u). Then r^: Orctj — » T is a well-defined contravariant functor from the category 
Ord^ of all finite ordinals and all mappings between them, and IXVj' 0 ) = p) n) for every 
map tl/. n) : 1 n with ^(O) =j G n. 

Let k be an abstract category and let H be an isomorphism of k onto a clone T on 
a non-empty set X. Then the composite or = // 1 o r> : Ord^ — ♦ k is a contravariant 
functor which endows k with the following additional structure: 

(g) k has a uniquely determined object a such that obj k = {a" \ n G a;}, where a" is 

an n-th power of a in the category k, and 
(e) k has a specific enumeration of product projections 7rj w) : a" — ► a by members of 
n. 

Indeed, it suffices to set a = a(l), a" = a(n) and 7rj' ,) = a^p) for y G n G w. 
Conversely, the information provided in (g) and (e) gives rise to such a contravariant 
functor a: Ord^ — » k. 

A category k for which (g) and (e) hold (or, equivalently, a category accompanied by 
such a functor a: Ord^ — » k) will be called an abstract clone and a G obj k its generating 
object. 

Therefore, up to trivial cases when a is not faithful, any abstract clone is the categor- 
ical dual of an algebraic theory as defined by Lawvere. 

We say that abstract clones k and k! are isomorphic, and we write 

k ~ k', 

if there exists an isofunctor <D of k onto k' such that <D o a = or 7 for the respective 
contravariant functors a and o/ of CmxL into it and Id. In particular, any isomorphism 
preserves generating objects and projection enumeration. 
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1.3. A pair (k, F) is called a clone whenever k is an abstract clone and F: k — ► Set is a 
faithful functor which preserves finite products. 

We say that clones (k, F) and (k 1 ', F) are isomorphic, and we write 

whenever the abstract clones k and kf are isomorphic. We say that the two clones are 
equal, and we write 

(*,F) = (A/,f v ), 

if there exists an isomorphism <t> of the abstract clone k onto the abstract clone k! (that 
is, an isofunctor <t> such that <D o a = a 1 ) for which Fo<S> = F. 

1 .4 . If X is a category with finite products, then every object a of X determines an 
abstract clone k, uniquely up to an isomorphism, as follows: for every nGwwe select 
an n-th power a" of a and an enumeration of its product projections, and then let k be the 
full subcategory of %i generated by these powers. 

If (30 U) is a concrete category such that 7C has finite products and the forgetful 
functor U: 3C — ► Set preserves them (that is, if (30 U) has finite concrete products), 
then every object a of 3£ with U{a) ^ 0 determines, up to an equality of clones induced 
on the non-empty set V{a), a clone {k, F), namely that consisting of an abstract clone 
k determined by a in 3£ and the restriction F of U to k. Any such clone (k, F) will be 
denoted as Clo(a), or Clo(a, 30 when 3C needs to be explicitly mentioned. 

1 .5 . Any nonvoid universal algebra A of a finitary type I determines two distinct clones 
on its underlying set X. As suggested by [6], for instance, these are: 

(1) the clone C\o(A) formed by all homomorphisms between finite powers of A, 
called the centralizer clone of A, and 

(2) the operation clone of A, which is the smallest clone on X containing all opera- 
tions of 4. 

While the operation clone of A determines the centralizer clone C\o(A) uniquely, its 
abstract clone does not do this at all, as shown in Theorem 1.9 below. 

1 .6 . Let r > 0 be an integer. If k is an abstract clone generated by the object a, then 
its full subcategory generated by {a 7 | j = 0, 1, . . . , r} C obj k augmented by the enu- 
meration of the product projections 7r, . al —* a inherited from k is called the r-segment 
of k. A pair (k r ,F r ) is the r-segment of a clone (k,F) if k r is the r-segment of k and F r 
is the restriction of F to the full subcategory k r of k. For any concrete category (30 U) 
with finite concrete products and any a € obj with U(a) ^ 0, the r-segment of a clone 
Clo(a) will be denoted by Clo r (a). 

We say that two r-segments k r , k! r of abstract clones k, k! are isomorphic and write 

if there exists an isofunctor <D r of k r onto k! r such that <P r o a r = a£ where a r , a' r are 
restrictions of the respective contravariant functors a and a!. 
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If (k r ,F r ) and are r-segments of clones, we say that they are isomorphic and 
write 

(*„/>) -(*;,/<;) 

whenever k r ~ \d r . Finally, the two clone r-segments {k r ,F r ) and (#,r>) are equal, and 
we write 

(kF r )«(£f r ) 

if there exists an isofunctor O r of k r onto such that O r o ct r = a£ and f y r oO r = F r . 

1 .7 . In addition to their equality and isomorphism, we shall also investigate elementary 
equivalence of clones and clone segments. We say that clones or r-segments of clones 
{k, F) and (k 1 , F 1 ) are elementarily equivalent, and write 

if the abstract clones or r-segments of abstract clones k and A ' are elementarily equivalent 
(in symbols k » K), that is, whenever it and k! satisfy the same formulas of the first order 
language of clone theory or of its fragment appropriate to r-segments. Paragraph 1 .8 
below introduces this language, as used by Taylor [8], and then in [9] and [7]. 

1.8 . Any abstract clone k with a generating object a can be viewed as an u-sorted 
universal algebra whose carrier X„ of the w-th sort is the set k{a n ,a) of all £-morphisms 
a" — > a, and which has 

(c) n distinct nullary operations cj 0 , . . . , cj?, of each sort n G uj, and 

n times 

(s) for any m, n G oj, an operation SJJ, of heterogeneous arity (n, m, . . . , m) whose 
values are of the sort m, that is, 

n times 

ST m :X n xX m x-- xX m ^X m . 

Indeed, we need only interpret c%\ . . . ,c^, as the product projections ttq\ n*J^, of 
k and write 

ST m (h,fo, ...,/,_,) = /i o (fix • */,_,) 

in terms of the composition in k — where fox - • xf„-\ is the unique £-morphism / in 
k(a m ,a n ) for which ^ of = f t for all /' G n. These operations also satisfy all equations 
below, namely, 

(ru> ss(A,4"\ • • • ,cf-i) = * for ever y » € ^ 

(LU) Si(cfVo. ■ ■ • ,/»-■) = J? for all m,n£u and / € n, 

(AC) 5P(A,S:(go/o,. ..,/„-!),. ..,5;(&,-i,/o,. •.,/»-!)) = s;(sS(/»,go,.-.,gp-i), 
fo,...,f„-\) for all G w. 

Hence every abstract clone it determines a unique assorted algebra whose operations 
are described in (c) and (s), and which satisfies equations (RU), (LU) and (AC). Con- 
versely, any such u;-sorted algebra determines, up to an isomorphism, an abstract clone k. 
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To see this, we choose an abstract generating object a for k and then formally require that 
k(a m , a") consist of all /i-tuples of members of the carrier X m of the sort m [with the unique 
'empty' 0-tuple in case of n = 0]. If a 'composite' g of off = (/o, . . . ,f n -\) € k(a"',a") 
and g = (go, . . . , g p - 1 ) 6 k(a n , of) is defined by 

8°f= (^migojo, • ■ ■ ,fn- 1 ), ■ • , ^ m (gp-\,fo, • • • ,fn- 1 )), 

then the associativity of o follows from (AC). The n-tuple (c { 0 n \ . . ..cJ2|) is the unit in 
k(a n ,a"), and a" is the n-Xh power of a with the projections c 0 n \ • • • € X„ = k(a n ,a) 
because of (RU) and(LU). 

The first order language of clone theory is nothing else but the first order language 
of the assorted algebras described above. Let r > 1 be an integer. Having restricted the 
sorts to {0, 1 , . . . , r} and the operations to those c| and for which / G n and m,n < r, 
we obtain a fragment of the first order language of clones which is just the first order 
language of clone /--segments. 

Given an abstract clone or an r-segment of an abstract clone k, we shall call the above 
assorted or (r + 1 )-sorted algebra the clone algebra of*. The carrier of its n-th sort, that 
is, the set k(a\ a) of *-morphisms, will be denoted by (k)„. 

It is clear that two abstract clones or clone segments are isomorphic exactly when 
their corresponding clone algebras are isomorphic and, more generally, that product pre- 
serving functors that also preserve product projection enumeration correspond exactly to 
homomorphisms of the corresponding clone algebras. 

1.9. MAIN THEOREM. If m,n y p e {l,2,...,oo} are such that m < n < p, then 
there exist finitary algebras A\ and A 2 whose centralizer clones Clo(/li) and Clo(^2> 

satisfy 

m = sup{/ < u | Clo,-(i4|) = Clo.042)}, 
n = sup{/ < ui | Clo,^,) ~ Cl 0l (^ 2 )}, 
p = sup{i < ui | Ck*(4i) « ClOi(A 2 )}. 

The algebras A\, A 2 have isomorphic operation clones. 

Next we outline the contents of the paper. In the second section, we assign a finitary 
algebra A(*), called a trace of X, to any metrizable topological space X. We also intro- 
duce the notion of a disciplined (metrizable) space X, for which we demonstrate a close 
connection between continuous maps X" -+ X and homomorphisms (h(X))" \(X). 
Then (see 2.9) we deduce that, for any two infinite disciplined spaces X U X 2 , 

C\o k (Xi) = C\o k (X 2 ) if and only if Clo*(A(*i)) = C\o k (\(X 2 )). 

In the third section (see 3.5), we prove that 

Clo A (*.) ~ Clo*(* 2 ) if and only if Clo*(A(*,)) a Clo A (A(^ 2 )) 
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for any infinite disciplined spaces X\ , X 2 with no absolute constants — a clone notion 
introduced in 3.2. In the fourth section on elementary equivalence (see 4.1), we show 
that such spaces also satisfy 

Clo t (Jr,) w Clo k (X 2 ) if and only if Clo*(A(*,)) » Clo k (\(X 2 )). 

In [7], we presented a general construction method that produces metric spaces whose 
clones of continuous maps have certain useful properties, and then applied it to obtain a 
result similar to the present Theorem 1.9: 

[7] for any choice of m, n,p £ { 1, 2, . . . , oo} with m <n<p, there exist metric 
spaces X\ andX 2 such that 

m = sup{/' < u | Clo ( (I|) = Clo,(* 2 )}, 
n = sup{/ < u | Clo/(X,) ~ Clo,(* 2 )}, 
p = sup{/ < u | Clo ( (^) » Clo,(* 2 )}. 

In the fifth section, we briefly describe this method, and then supplement [7] by show- 
ing that all spaces it produces are disciplined, and almost none of them have absolute con- 
stants. Together with the results of Sections 2-4 mentioned earlier, this already shows 
that the algebras^ = A(*0 and/l 2 = A(* 2 ) satisfy the first statement of Theorem 1.9. 
Its remainder— the claim that the operation clones of A\, A 2 are isomorphic — is proved 
in the sixth section, where we show that traces of any two infinite metrizable spaces sat- 
isfy precisely the same set of identities. This implies that operation clones of algebras 
A\ = A\(m,n,p) and A 2 = A 2 (m,n,p) are isomorphic for all choices m < n < p in 
{1,2, . . . ,oo}, so that they determine the same abstract clone. Moreover, all these al- 
gebras will be constructed over the same underlying set. This means that an abstract 
(operation) clone can have representations on the same set which vary to the extent that 
their centralizer clones realize every triple m < n < p in {1,2,..., oo}, in the sense 
described by Theorem 1 .9. 

Using [7] as a source of disciplined spaces without absolute constants, in the sev- 
enth section we apply the trace to 'translate' other results of [7] into results on finitary 
algebras. This section also describes alternate traces (unary and infinitary), and restates 
Theorem 1 .9 for a finitary algebra and its reduct. 

2. Disciplined spaces and their algebraic traces. 

2. 1 . To any metrizable topological space X = (P, r) we now assign its algebraic trace 
&{X). This is a universal algebra A(A^ = {Q, {-,7} U{n k \ke a>}) with a single binary 
operation • and countably many unary operations 7r 0 , 7Ti , . . . and 1. The underlying set Q 
of A(JQ is the disjoint union 

0 = PUSU{A}, 
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where the set S consists of all one-to-one sequences {pk | k G u)} of members of P. The 
operations of A(A) are defined as follows: 



It is clear that A(A) is isomorphic to ^(X 1 ) whenever X is homeomorphic to X 1 . Con- 
versely, if the traces A(A), A(A*) are isomorphic then X is homeomorphic to X. Indeed, 
if A(A) is a singleton then X = X = 0. Else P C Q consists of all q G Q with q- q = q 
satisfying q q' ^ q for every q' ^ q, and hence P is bijective to the underlying set P 1 of 
X 1 , and X is homeomorphic to X 1 because of the remaining unary operations. 

2.2. Definition. Let n G tit. We say that a topological space * = (P,/) is /i- 
disciplined if every continuous map /: X" — » A' decomposes as a projection followed by 
a homeomorphism onto a closed subset of A' or, more precisely, if for some set M C n = 
{0, . . . , /i - 1 } and some homeomorhism g: A** — > A" of A** onto a closed subset of A', 



where A" A^ is the projection of A" onto X M associated with the inclusion map 

MCn. 

A space X is disciplined if it is w-disciplined for every nGw. 

REMARKS, (a) It is clear that every T\ -space is 0-disciplined. 

(b) A space X is /i-disciplined if and only if for every continuous map /: X" — » X 
there exist an integer A: > 0, a one-to-one mapping rp:k —+ n, and a homeomorphism 
g:A* -> A" onto a closed subset of A" such that f = go tP>\ where it*hX* -* A* 
is the (surjective) projection given by 7r lv,] (y>) = ip o tp for every y? G A 71 . Any such 
decomposition / = g o ?rM will be called standard. 

For any integer /i > 0, let A (w) : A(Ay — » A(A) be the constant map whose value is A. 
Here is the central result of this section. 

2.3 . PROPOSITION. Let A(A) be the algebraic trace of a disciplined metrizable space 
X=(P,t).andletn>0. Then 

(I) every continuous map f \X* — > X has a unique extension to a homomorphism 
£(/): \(Xy — A(A), and E(f) is constant if and only iff is constant, 




q if q = q' G P 
X otherwise; 



for every k G uj, we set 




and finally, the operation 7 which traces the topology t of X is given by 



p ifqES and </ converges to p in A 
7(<7) = < A if <jr G S does not converge in X 
q for every q G PU {A}. 



f=go*M 
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(2) ifh: K(Xf \(X) is a homomorphism, then either h = A (n) , or else h(P") C P 
and the restriction ofhtoP" maps X" continuously into X. 

We recall that the algebraic trace A(^) = (0,{-,7}U{7r y \j G u;}) of a space* = (P,f) 
is defined on the disjoint union Q = PU SU {A}. For k > 1 and an arbitrary one-to-one 
map g: P* — » P we define a map 

as follows. For q = (q 0 , . . .,qk-\) G Q*, we set 

.... , . . [ A if at least one q t = A 

(hi) h g (qo,...,q k -0={ g(q) {fqepk 

The case still left undefined is that of a *-tuple q £(PUSf with qi € S for at least one 
/ 6 k. For any such A>tuple (q 0 , . . . , q k ~ \ ) and every /' G £ and we first write 



* = { 



when qi G P 
when G 5, 



and then assemble a sequence {qy | y G u>} with the members q) = (fy.o, . . . ,q"jjc-\) G 
P* C 0*. This sequence is one-to-one because qi G S for some / G £. Since the mapping 
g is one-to-one, 

(h2) ...,«m)- a;} 

is a one-to-one sequence of elements of P. Whence h g (qo, . . . ,qk-\) G S, and (h2) and 
(hi) define a mapping h g :Qf — > Q correctly and completely for any k > 1 . 

When k = 0, we have P° = 0° = {0} and, for any g: P° — ► P, we define h g :QP->Q 
to be the map with h g (<d) = g(0). 

Proposition 2.3 is proved in 2.4-2.9 below. 

2.4. LEMMA. Z,e/ k>0bean integer and let g: P* — » Pbea homeomorphism of X k 
onto a closed subset of X. Then E(g) = h g : &(X? — > A(A^ is a homomorphism. 

PROOF. Write A g = h. Since {/?} is a subalgebra of P±(X) for every p G P, we need 
consider only the case of k > l.Letq = (qo,...,q k - { ) and = (q' 0 , ...,q' k _ x ) belong 
toC2*. 

a) First we show that the binary operation • is preserved by h. 

al) If q ^ q', then q t ■ q\ = A for some ; G so that h{q ■ q') = A. We show that 
%) • h(q') = A as well. If G SU {A} for some / G k, then %) G SU {A} and hence 
h(q)-z = A for all z e Q. Since the operation • is commutative, in the only remaining case 

we have q,q' G P*. But h coincides with g on P* and g is one-to-one, so that h(q) ^ /»(<?') 
and %) A(g / ) = A follows. 

a2) If q = q', then either q, G SU {A} for some / G * and % ■ q') = A = A(<7) • M?') 
as in the previous case, or q G P\ and hence 9 • 9 = 4. But then h(q • g) = gfa) and 

%) %) = gfe) g(<7) = giq). 

b) Next we prove that h preserves tt, for each j G 
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bl) If q G (PU {A})*, then iryfo) = q t for all » G k, and hence rryfa) = q. Also, 
%) € PU {A} by (hi), and tt,(%)) = %) = %(?)) follows. 

b2) If?, = A for some/ G *, then tt,^,) = A^othat/^)) = A = %) = irj(h(q)), 
by (hi) again. 

b3) In the remaining case of a q G (P U S)* with qi £ S for some / G Ar, the A>tuple 
r = (7r,(qr 0 ), • • • , 1 )) is such that Kj(qi) = ?/ when q t G P, and Wj{qi) G P is the y-th 
member of the sequence qi when q t G 5, so that r G P* and /»(r) = g(r) = Ttj(h(qj) by 
(h2). 

c) Finally, we show that h(l(q 0 ), . . . 7(?*_i)) = 7(%)) for all q = (^ 0 , • • • , G 
£?*. This is clearly true for q G (P U {A})* because 7 is the identity on P U {A} and 
h(q) G PU {A} for any such q, and also for any A>tuple q with some q\ = A, for then 

A(7(<7o), • • ,7fafc-i)) = A = 7(A) = 7(%)) by (hi). 

In the remaining case, we have aq G (PUS)/ 1 with qi £ S for some / G A:. As in 
the clause (h2) of the definition of h, let q = {qj \ j G u;} be the one-to-one sequence 
of elements q } = (q jfit . . . ,q jt k-\) of P*, such that qjj = q t when G P, and is the 
y-th member of q, when q { G 5. First, let us suppose that q converges to a point r = 
(r 0 , . . . , />_,) in the space )& = (P, /)*. Then, for every I G *, the sequence | y G w} 
converges to r, in X, so that (7(90), • ■ • , 7(f*- 1 )) = r, and hence /»(7(<?o), • • • , 7(?*_ 1 )) = 
A(r). On the other hand, h(q) = {g(qj) 1 7 € w} € 5 by (h2), and the sequence {g(^) | 
j G a;} converges to g(r) because g is continuous. But h(r) = g(r) since r G P*, and 
therefore 7(A(^)) = /i(r) = h{l(c[o), . . . ,7(<7*-i)). Secondly, let us suppose that q does 
not converge in A*. Then there must be a sequence {qp 1 7 G u>} which does not converge 
in X. Whence l{qjj | j G a;} = A. Simultaneously, since g is a homeomorphism of P* 
onto a closed subset of (P,/), the sequence (g(<7 7 ) | y € w} = h(q) cannot converge in 
(P, t) either. Therefore 7(%)) = A = h{y(q 0 ), . . .,7fa*-i)). ■ 

2.5. LEMMA. Z,e/ k>0bean integer and let h: A(AQ* — » &(X) be a homomorphism. 
Then either h is constant with the value in P U { A }, or else A(A, . . . , A) = A, h(J*) C P, 
and the restriction ofhtoF* is not constant. 

PROOF, (a) Sincez- A = A forallz G Q, forevery q G £?* wehaveg • (A,. . . , A) = 
(A, . . . , A), and hence also h(q) • h(X, . . . , A) = h(\, . . . , A). If h(\, . . . , A) ^ A, then 
h(\, . . . , A) G P and hence h(q) = h{\, . . . , A) for all q G Q*. Therefore h(X, . . . , A) = A 
for any h which is not a constant whose value belongs to P. 

(b) Suppose that p = (po, . . . ,pk-i) G P* and A(/?) = A. We aim to show that h must 
be the constant with the value A. Since 7(z) = z exactly when z G PU {A}, the homo- 
morphism h must map (P U {A})* into P U {A}. First we show that h(p') = A for any 
other p' = (p' Q , . . . y p' k _ x ) G P*. For / G k, we set q> = p, when p\ = p t and, when p\ ^ p„ 
choose q { G 5 with 7r 0 (^,) = p { and 7Ti(^,) = p\. Then (7r 0 (^o), • ■ , 7r 0 (^_i)) = p and 

(tti(<?o), - - - , tti(^a-i)) =/?'. Whence 7r 0 (%o,--W*-i)) = ^(^o^o ),•••, *o(qk-\)) = 
h{p) = A. But then h(qo,. . . ,q k -\) = A because tvq ] {\} = {A}, and h(p') = 
7T|(/j(<7o,-..,4*-i)) = 7T((A) = A follows. Therefore /i(P*) = {A}. Next we show 
that h(q) = A for every q = (q Q> . . . ,q k -\) G (P U {A})*. For any such q, there exist 
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p = (p 0 , . . . ,p k -\) and p' = (p' 0 , . . . ,p' k _ x ) in P* with q { = p { • p\ for every / G k, and 
hence h(q) = A(p) • h(p') = A • A = A. Finally, for any q = (qo, . . . , q^-x) G @* we have 

(tto(<7o), .- ,^o(ft-l)) € (PU{A})*, sothat 7r 0 (%)) = /»(tt 0 (^o) 7r 0 (^*_ i )) = A. 

Since ir 0 (z) = A in A(A) only when z = A, this proves that h(q) = A. Whence h is the 
constant with the value A. 

If h is not constant, then (a) and (b) show that h(X, . . . , A) = A, h(P k ) C P, and that 
the restriction of h to P* is not constant ■ 

2.6. LEMMA. Letk>0 be an integer and let h\,h 2 : A(AY — > A(A) be homomor- 
phisms such that h\(p) = h 2 (p) for every p G P*. Then h\ = h 2 . 

Proof. Since the claim is trivial when k = 0, let us assume that * > 1. By 
Lemma 2.5, the homomorphisms h\ , h 2 agree on P* U {A}*. 

(a) To show that h\(q) = h 2 (q) for each q = (<7o, • • • ,<7*-i) € (PU {A})*, we select 
p = (p 0y ... ,p k _i) and p' = (p' 0 ,... ,p' k _i) in P* so that/?, • = for every / 6 and 
conclude that h x {q) = /ii(p) • *i(pO = h 2 (p) ■ h 2 {p') = h 2 (q). 

(b) Let 9 = (^o,...,^-i)ee t .Then(7r ; (^o),...,7r / (<7*_ 1 )) G (PU {A})* for every 
y G The maps A,, h 2 satisfy *y(A|(9)) = /ij (79(^0), ■ • .^j(qk-\)) and 7Ty(A 2 (g)) = 
h^jiqo), TTjiqk-])). But then -Kj(h\(q)) = -Kj(h 2 {q)) for every j £ u because h\ , /j 2 
agree on (PL) {A})*, and h\(q) = h 2 (q) follows. ■ 

2.7 . Now we are in a position to prove Proposition 2.3( 1 ). Let A' = (P, /) be a disciplined 
metrizable space, and let /: X" — » A' be continuous with n > 0. Since X is disciplined, 
we have a standard decomposition f = go of /, where 7r^ : X" — > A* for some k > 0 
and an injective — > «, and g:A* — > A" is a homeomorphism of A* onto a closed 
subset of X. Then the homomorphism /i g : &(Xf — » A(A) from Lemma 2.4 extends g. 
Whence E(f) = h g o 71-M, where tt^I denotes the projection A(A) fl -+ A(A)* associated 
with ip, and £(/) is a homomorphism extending /. Its uniqueness follows by Lemma 2.6. 
The homomorphism £(/") is constant exactly when / is constant, by Lemma 2.5. This 
completes the proof of Proposition 2.3(1). 

Proposition 2.3(2) will follow directly from Lemma 2.6 and the Lemma 2.8 below. 

2.8. LEMMA. Letn>0 be an integer, and letX = (P, /) be a metrizable space. Then 
any homomorphism h: A(A)" — » A(A) other than \ (n) is an extension of a continuous map 
f:X«-*X. 

Proof. If h ^ A (n) then, by Lemma 2.5, either h is a constant whose value belongs 
to P, or else h(X, . . . , A) = A and h(P") C P. Thus we need consider only the latter case 
for* > 1. 

Suppose that the restriction f :X" — > X of h to P* is not continuous at some r = 
(r 0 , . . . ,r„_0 G P". Then there exists a sequence {v, \j G u} which converges to r in X" 
and such that, for some e > 0, the distance from /(v,) to f(r) is never less than e . If we 
write vj = (vy.o, . . . tyn-i) for each j G u, then the sequence v* = {vjj | j G cj} converges 
to r, for every / G n. We need to produce a sequence {wj \ j G u} in P" amenable to 
algebraic arguments, that is, a sequence whose coordinate sequences {w Jti \ j G uj} are 
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either one-to-one or constant for / £ n. We proceed as follows. Any sequence contains 
a subsequence that is either constant or one-to-one. There exists a subsequence {vj | 
j £ u)} of {vj ■ \ j £ u;} whose first-coordinate sequence {v? 0 | j £ a;} is one-to-one or 
constant. Then we find a subsequence { vj \ j £ uj} of {vj \ j £ u} such that the sequence 
{vj , \j £ u} of its second coordinates is one-to-one or constant, and repeat this process 
until, having exhausted all n coordinates, we arrive at a subsequence {vv, | j £ uj} of 
{vj | j £ a;}. If Wj — (wjfl,. . . ,Wj t „-i) and / € n, then the i-th coordinate sequence 
W = {wjj | j £ w} is either constant with wjj = n for all j £ u, or else it is one-to-one 
and converges to r,. 

For each i £ n, we choose q t £ Q as follows: 

(w* if w* is one-to-one 
r, if W is constant. 

Then ^, £ S in the first instance, £ P in the second and, moreover, 7(<7,) = r, and 
7l >(^/) = w j,i f° r a ^ ' £ " an d / £ Thus, in particular, q = (q 0 , . . .,q n -\) £ (PUS)" 
and (7(^o), • • .,1(q„-\)) = r in A(^y. Also, for every y £ u>, 

because A(P') C P. From 7rr'(P) = PU S it follows that either h(q) £ P or %) £ S. In 
the first case h(q) = f(wj) for all j £ a;, while %) = {f(wj) | y £ a;} £ S in the second. 
But then 

= h(l(q&...,l(«-l)) = Kr) =f(r), 

and this is possible only when either fiyvj) = f(r) for all j £ a;, or {f(wj) \ j e u} 
is one-to-one and converges to f(r). But this is impossible since {fiyvj) \ j £ u} is a 
subsequence of the sequence {/"(v,-) | j £ a;} which does not converge to r. ■ 
The proof of Proposition 2.3 is now complete. 

2.9. PROPOSITION. Let k>0 be an integer and let X\ and X 2 be disciplined metriz- 
able spaces. Then 

C\o k (X x ) = C\o k (X 2 ) if and only if Clo*(A(X,)) = Clo*(A(* 2 )). 

PROOF. This is an immediate consequence of Proposition 2.3. ■ 

3. Absolute constants in disciplined spaces. 

3.1 . Our Theorem 1.9 speaks about the equality =, isomorphism ~ and elementary 
equivalence ^ of centralizer clone segments Clo,(/l 1 ) and Clo,^) of finitary algebras 1 
and/42 which, as indicated earlier, are the algebraic traces of suitable disciplined metriz- 
able spaces. The equality of centralizer clone segments of such algebras has already been 
treated by Proposition 2.9. Here we investigate their isomorphism, for which we prove 
a similar Proposition 3.5 and, in 3.8, also set down a basis for subsequent consideration 
of elementary equivalence. 
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We need the following concepts. 

3.2. Definition, (a) Let F:k -» Set be a functor and let a be an object of the 
category k. We say that an element x G F(a) is an absolute fixpoint of a whenever, for 
any moronism / G k(a, a), either the mapping F(f) is constant or F(f)(x) = x. 

(b) Let a e objk generate a clone (k,F), and let n G u. A morphism c G k(a",a) 
is called an absolute constant whenever F(c): F(a n ) — » F(a) is a constant map 
whose value is an absolute fixpoint of a. 

3.3 . We recall that the first order language of clones is just the first order language of 
cu-sorted algebras with the operations cj n) and S? m with j en and m,n Go;, as described in 

1 .8. The language has u sorts of variables, with symbols Jr^.y 0 , jc*,'*, xj*, . . . denoting the 
variables of the i-th sort for each i G uj. For any r > 1 , the first order language of clone 
r-segments is the fragment of the first order language of clones which has variables jc^, 
y°, . . . with i = 0, 1, . . . , r only, and operation symbols S? m and cj" ) with j en restricted 
to tn, n < r. 

Since in our case the variables of the i-th sort range through maps V — > K, we simplify 
matters somewhat by treating all variables as maps. As is customary, we shall identify 
elements of Y with constant maps Y — » Y, so that a variable will be replaced by a 
variable ^ of the 1-st sort satisfying the predicate 

p(x il) )=^ l) S\(^K/ l) ) = x il K 

In what follows, we shall also use the predicates 

afptf 0 ) = p(* (1) ) A Vy< 1 >(p(y< , >) VS^V 0 ) - * (,) ) and 

ac(;>>) = ^(alfcxW) A S!fr<V*) = 2<»>), 

which describe absolute fixpoints and absolute constants in this language. 

3.4 . If X = (P, /) is an infinite disciplined metrizable space then, by Lemma 2.5, any 
homomorphism h: &(X) — > h(X) with A(A) ^ A is a constant map (whose value belongs 
to P). Whence A is an absolute fixpoint of k{X), and the homomorphism A (n) : h(Xy — * 
A(A^ is an absolute constant in Clo(A(A)) for every n > 0. 

If A' has no absolute fixpoints, then for every p e P there is a non-constant continuous 
map f p :X -» A r with y^(p) ^ p. Hence the unique homomorphism \(X) -» A(A) 
extending^— see Proposition 2.3( 1 >— is non-constant and such that E{f p )(s) ^ s for any 
s G S with 7r 0 (s) = p. It follows that has no absolute fixpoints other than A, that A {n) 
is the only absolute constant &(X) n -* \(X), and that Clo(A) has no absolute constants. 

Hence, by 2.3, for every n G w, 

Clo(A(A0)„ = {£(/) |/G Clo(A)„}U {A ( ">}, 
and A (n) is the unique absolute constant in Clo(A(A)) n . Let 

£:CloW-»CIo(AW) 

denote both the embedding of these clone algebras and also the functor preserving finite 
products and projection enumeration corresponding to it. 
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3.5. PROPOSITION. Letk> 1 be an integer, and let X\ andX 2 be infinite disciplined 
metrizable spaces without absolute fixpoints. Then 

CIoaW) ~ C\o k (X 2 ) if and only if Clo*(A(*,)) ~ C\o k (\(X 2 j). 

This equivalence also holds for the respective full clones. 

PROOF. In the diagram below, let and EP be restrictions of embeddings from 
3.4 above to the respective clone segments. 

CM*,) ^ cio*(A(x,)) 

•I 2) I* 

C\o k (X 2 ) ^ Clo*(A(* 2 )) 

Suppose first that the functor ¥ in this diagram is an isomorphism. Then 4* preserves 
elements satisfying the same first order formulas, and hence it must map absolute con- 
stants of its domain precisely onto absolute constants in its codomain. By 3.4, there is a 
unique isomorphism Q> completing the diagram above. 

Conversely, let O be an isomorphism. We extend O to a product-preserving functor 4* 
which sends the unique absolute constant A (n) of Clo( A^ )) to the unique absolute con- 
stant of Clo(A(A r 2)) n for every n < k. Then the diagram above commutes. The fact that 
the bijection T is a clone isomorphism or, equivalently, that it preserves all operations 
SJJ, with m, n < k, will follow from Lemma 3.8 below. The remainder is clear. ■ 

3.6 . Let K = (k, F) be a clone with the generating object a, and let il>: r — * n be a 
map for some r,n € u). As in 1 .2 for clones of spaces, but now for any clone, we write 
7r lv,) : a" — ♦ a r to denote the £-morphism 

-M - JM x...xn (n) 

* 7r v(0) A A %<r-I)' 

Then, for any h G k(a r , a), 

follows from the definition of clone algebra operations. 

3.7 . Let K = (k,F) be a clone with the generating object a. Since F is faithful and 
a" ^ a m for distinct n, m e u, any *-morphism g € k(a",a) is fully determined by F(g). 
Hence there is no need to distinguish F(g) from g, and we shall write g instead of F(g) 
as well. 

Definition. Let g e k(a", a). We say that 

g = h o n M 

is an extremal decomposition ofg— in the clone K or in any of its m-segments with m > 
n — whenever ib: r — » n is a one-to-one map, and for any decomposition g = h\ o tt 1 ^ ] 
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with a one-to-one map rp\:s — » n there exists a A:-morphism d G Ha*,<f) such that 
</o7r^ = 7rM 

REMARKS, (a) Since t/> and xp\ in the definition above are both one-to-one, we have 
r,s < n. Thus the definition does apply not only to the full clone, but also to all its 
m-segments with m>n. 

(b) Any factorizing morphism d is unique and has the form d = -n^ for some 8 with 

(c) Since the map ir w is surjective for any injective map xj), an extremal decomposi- 
tion is determined uniquely up to a commuting isomorphism. 

Observation. If g = h o tt^ g k(a n , a) with tp: r «, and ifj0 G Ha m > a) for j G n, 
then 

Wo,-^-i) = go(/-oX-.-x/ fl _,) = /io (f m x • • • x/^ r _ , } ) 

= ^(M^O), - • 1))- 

This follows immediately from ^ = t^x ■ ■ • 

3.8 . The Lemma below summarizes the relationship of Clo(A(AT)) to Clo(X) in a form 
suitable for investigations of elementary equivalence. 

Let E: C\o(X) — ♦ C\o{h{X)) be the embedding from 3.4. Proposition 2.3 shows that, 
for each / G n, the homomorphism E(tt^) G Clo(A(A^) n is the i-th projection, and that 
£(7r5 n) ) uniquely extends the i-th projection T^ n) G Clo(A^„. It follows that E(tt w ) is the 
unique extension of ir^ = 7r^ 0) x • • • X7r^ r l) for any V>: r — ♦ n as well. This allows us 
to simplify the notation as follows: we write iff* instead of Ein^) G Clo(A(A0) and, 
more generally, *M instead of E(^) G C\o(\{X)). 

LEMMA. IfX = (P, t) is a disciplined metrizable space and n G u, then Clo(X) and 
C\o(h(X)) have the following properties: 

(1) Clo( A (*))„ = W) I g! e C\o(X) n } U { A<"> }. 

(2) For any gf G Clo(X)„, any standard decomposition g 1 = h' o tt w of g is also an 
extremal decomposition of g'. 

(3) Every g = £(g0 G Clo(A(J0) \ {A ( ">} has an extremal decomposition g = 
E(h') o where g' = h' o -n^ is an extremal decomposition of g 7 . 

(4) Letf = (/&*••• x/„-i): A(A v ) m A(A v ) n a/u/g: A(X) n -» A(A^) in Clo(A(A v )). 
77je« e/7Aer g = A (,,) ant/ Aewce go/= A (m) , or efce g /ia5 a/i extremal decompo- 
sition g = E(h') o 7r [vl a/jJ 

( A( m) if/ 0(O = A<"»> for some i G r 

g ° J \ E(h') o (/"^o) x • • • x/^ r _ ,)) otherwise, 

withf m = E(f m )for all i G r. 
77zese conclusions also hold for any initial k-segments of these clones when k>\. 

PROOF. Claim (1) is already contained in 3.4. To prove (2), let gf = h' o tt^ be a 
standard decomposition. To show that this is also an extremal decomposition, suppose 
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that g 1 = h\ o ifiM for some injective ip\:s —> n. Since the projections 7r lvl and tt 1 ^ 1 are 
surjective and h' is a homeomorphism of X r onto h\(X*), the composite d = o h\ 
exists and is continuous. But then ti o d o Tr^'l = A' o ttM, and rf^'l = tt^ follows 
because h' is injective. 

Now we turn to (3). If g ^ A (n) , then g = E{g!) for a unique g'-.X" —> X,by (\), and 
g 7 has an extremal decomposition gf = h' o 7r 1 ^, by (2). From Lemma 2.6 it then follows 
that £(7r tv 'l) = 7r 1 ^ andg = E(h') o To show that this is an extremal decomposition 
of g, suppose that g = hi o with an injective t/?i : s — » n. Then h\ ^ A (5) because 
g ? A« so that A, = E(h\) for some /»', G CloW*, by (1) again, and gf = h\ o 
follows. Since the decomposition gf = h' o 7^ is extremal, we have d o *Md = for 
some d G C\o(X), and hence E(d) o ir^' 1 = tt^ in Clo( A(A)) . Therefore g = E(h') o tt^ 
is an extremal decomposition of g, as claimed. 

To prove (4), letg ^ A (n) . Theng = E{g*) = E{h')o^ { s an extremal decomposition 
of g in which h' is a homeomorphism of A 7 ^ onto a closed subset of X, by (3) and (2), 
and gof = E(h') o (f m x • • • x/y< r _i)) by Observation 3.7. From the definition of the 
extension E(h') of the homeomorphism h' it then follows that gof = A (m) whenever 
f m = A< w > for at least one i G r. If, on the other hand,/* 0 ^ A (m > for every i G r, then 
/w) = £ (/Jk/)) for 3,1 ' G r by ( 1 ), and (4) follows. ■ 

4. Elementary equivalence. 

4. 1 . The purpose of this section is to verify the following claim about elementary equiv- 
alence of clones or their segments. 

PROPOSITION . For any two infinite disciplined metrizable spaces X x and X 2 with no 
absolute fixpoints, and for any k > 1, 

C1oa(*,) W C\o k (X 2 ) if and only if Clo A (A(A',)) « Clo*(A(* 2 )). 

This equivalence also holds for the respective full clones. 

Even though it is easy to show, using Lemma 3.8 and a natural uniform interpretation 
of sentences from the theory of Clo*(A0 in the theory of Clo A (A(A)), that Clo*(A",) W 
CIoa(* 2 ) follows from Clo*(A(A',)) « Clo*(A(^ 2 )), sentences claiming the existence 
of absolute constants make a converse interpretation unwieldy, and we employ other 
means to prove the above Proposition. Specifically, we use the well-known fact that el- 
ementary equivalence is equivalent to isomorphism of prime limits (see [4] for mono- 
sorted algebras). We prove that an isomorphism of certain prime limits of CIo*(*,) and 
Clo*(* 2 ) can be extended to an isomorphism of associated prime limits of Clo*(A(*,)) 
and Clo*(A(X 2 )) and, vice versa, that a restriction of an isomorphism of some prime 
limits of Clo A (A(*,)) and Clo*(A(* 2 )) is an isomorphism of the associated prime limits 
ofClo*(*,)andClo A (X 2 ). 

4.2 . In order to introduce an appropriate notation, we recall how ultrapowers of clone 
algebras are created. 
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Let £ be a clone algebra as described in 1 .8, and let K„ denote its carrier of the «-th 
sort. Given an ultrafilter J on a set /, for any s, t G K' n we define 

s ~j tiff Eq(s, 0 = {/ G / I = G 7. 

Then ~<f is an equivalence. For any s G K l „, the symbol [s]j will denote the class of 
~j containing s. If k (7) is the natural map of K' n onto the ultrapower KS/ ) = K l n / ~j 
with the kernel ~<f then «|^(s) = [s], so that each a G /^^ ) may be replaced, as is 
customary, by any s £K l n with [s]j = a. Whence t G [s]y exactly when Eq(s, t) G 7 ■ 

For any w G K n , we let vv G K' n stand for the constant sequence whose value is w. For 
each € n we then write 

(<fjt* = G *</>, 

where <*> is the nullary operation from 1 .8(c). 

Let (Sr m Y: K'„ x {K' m ) n — » ATj, be the componentwise extension of an operation S^,: K n x 
A^J, — > A m from 1.8(s), that is, let 

((S^fts, f 0 , • • • , '„-i))(0 = <o(0, • • • 

for every / G /, any s G K n and any fo, . . ■ , '„-i G K m . If s' G [s] j and /J G tyly for all 
j G «, then 

Eq(5,5')nn{Eq(/ y ,(;)|yG/i}€ J, 

and hence also 

Bq((£/<Mb..., OSDV, C— • Ci)) e 7". 
It follows that there is a well-defined and unique mapping 

given for a G Kj^* and r 0 , . . . r„_ i G KS 7) by 

(S^Wo, . . . ,r fl _,) = [(5^/(5, / 0 , . . . , j 

with an arbitrary choice of s G a and (, G tj for each y G n. 

Let — * be the diagonal map given by \(k) = [k]j G KS 7) for all A: G 
K„. Then x is a one-to-one homomorphism of the clone algebra K into a heterogeneous 
algebra KS 7> > whose operations (cj/ 0 )^ and (^) (!F) satisfy (RU), (LU) and (AC) from 

1.8. Having identified every k G K with its image \(k) G we thus obtain a clone 
algebra tf^ whose carrier of the rt-th sort is KSp and whose operations are {cff^ = 
cj n) and (ST m ) {:F) . The clone algebra KS^ will be called a prime power of K associated 
with f. 

Let K, L be clone algebras, and let E: K — » L be a clone homomorphism. The definition 
of its extension ES 7) :KS^ — is evident, and it is clear that El?) is an embedding 
whenever E is an embedding. 
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4.3 . For any integer r > 1, let f r be an ultrafilter on some set l r and let F = {% \ 
r > 1}. Set K(0) = K, then inductively define K (r ) = for each r > 1, and let 
X r :/Q r _D — * K( r ) denote the diagonal embedding arising at each step. If 0 < s < r, 
then \ r o • • • o \s+\ is an injective homomorphism, and we may therefore assume that 
K( S ) is a subalgebra of K {r) whenever 0 < s < r. These composite embeddings, or rather 
inclusions, form a diagram whose colimit PLf(/Q is an assorted algebra isomorphic to 
the union of the u;-chain 

A(0) >K( 2 ) ► • • • , 

with colimit injections p. s :K( S ) —* PLr(^0- It is clear that the assorted algebra PLp(/Q, 
called a prime limit of K over F, satisfies (LU), (RU) and (AC) from 1 .8 with the nullary 
operations cj n) € K {0) for all j En and joint extensions (S" m ) (¥) of all (S%,) l ? r) with r > 1. 

For a clone homomorphism E:K —> L, the definition of PLf (£): PLf(AT) — » PLr(L) is 
evident. It is clear that if E is an embedding then PLf(E) is also an embedding. 

4.4 . Let X be an infinite disciplined space with no absolute fixpoints, and let F be a 
sequence of ultrafilters. In the lemma below, we extend claims (l)-(4) of Lemma 3.8 
about the clone algebras C\o(X) and Clo(A(A^) to analogous claims about their prime 

limits A = PLf(CloW) and/T = PLf(c1o(A(*))). 

The non-nullary clone operations will be denoted as S% \x\A, and as (5^)* in A*, while 
c) n) will denote the clone constant corresponding to the product projection ir\ n) in C\o(X) 
and in C\o(&(X)), in both A and A*. 

From 3.7 we recall that any existing extremal decomposition is determined uniquely 
up to a commuting isomorphism. We note that, in the lemma below, all decompositions 
are expressed by means of clone algebra operations, see Observation 3.7. 

LEMMA. Let X, F.A.A* be as above. Then: 

(1) There exists a one-to-one clone homomorphism E.A — ► A* such that the carrier 
A* of the n-th sort of A* is the disjoint union A* = {Eig 1 ) \ g* e A n }U {* (n) }, 
and *( n) is the only absolute constant in A*. 

(2) Each g' £ A„ has an extremal decomposition g 1 = ST„(h', c ( ^ 0) , . . .»c£L|A 

(3) E:A —* A* preserves extremal decompositions, that is, if g € A*, g = Eig 1 ) 
and g 1 = S^A'.cJj^, • • " an extremal decomposition of g 1 in A, then 
g = (S£)* (£(*'), cjjij, . . .,cj$_i)) is an extremal decomposition of g in A*. 

(4) Forfr, ...,/„-i 6 A* m andg G A*, denote 

c = (sr m )*(gJo,...,f n -\). 

Then C = * (m) for g = * {n) . Otherwise g € A* \ {* {n) } has an extremal decom- 
position g = (Sr n Y(h, c$ or . . . , c%_ . by UH3). and 

r _ J *< m > iff m = *<"> for some i € r 

" 1 (^) W«o>, ■ ■ ■ ,/v<,-») Vf m ¥ * im) for all i e r. 
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The same conclusions hold for prime limits PLf(Clo*(*)) and PLf (cio*(A(*))) ofk- 
segments of the clone algebras. 

Proof. Denote A {0) = Clo(A"), A[ Q) = CIo(A(*)), and let £ (0) : A (Q) — > /1 ( * 0) be as in 

3.4. For every s > 1, write ^ = <4*-lj)f*'\ ^ = (>!*,_ and £ w = (£(,_,))<*>, 
where ^ G F is an ultrafilter on a set 

From 3.4 and Lemma 3.8 it follows that A {0) , A\ Q) and E {0) already satisfy (1 ), (2), (3) 
and (4). From the definition of a prime limit it follows that the lemma will be proved 
once we show that these four statements hold for A {s) , A* (s) and E (s) whenever they hold 
for^-i), and 

Hence, let an ultrafilter f on a set / be given. We shall suppose that ( 1 H4) hold for 
clone algebras K,L = K* and a clone-embedding E:K—>L, and prove these statements 
for the ultrapowers KS^\ and ES?\ To simplify the notation, let us suppose that the 
embedding E is an inclusion, so that A: is a subalgebra of L and each operation (S" m Y of 
L is a mere extension of the operation SJJ, of K. 

To prove (1), assume that the carrier L„ of the n-\h sort of L is the disjoint union 
L„ = K n U {* (n) }, in which * (fl) is the only absolute constant. Clearly KSp C L { /\ 
Furthermore, the sentence Vx^-iacC**"*) holds in K„, while its negation 3x* n) ac(x^ n) ) is 
satisfied in L„ by * (n) alone. For an arbitrary / G L)?\ the set / is the disjoint union of 
J={i£l\ f{i) G AT*} and / = {iel\ f(i) = * (n) }, so that exactly one of these sets 
belongs to ^ If 7 € F, then / G A^/> and / is not an absolute constant. Else J' G 7 
and / ~j and / must be the absolute constant in tijp. Having identified diagonal 
members of Up with their values in L n , we obtain (1) for tf/> and L^. 

For (2), select any g G Kj*> Then J = {/ G / j g(i) G AT,,} G J. Since g(0 ^ 
* (fl) for all i G ./, every such g{i) has an extremal decomposition g(i) = 
^(^)»c£|o)f«''»*$^j-i)) » n Each V>;:r,- — » « is injective, there are only finitely 
many of these maps, and hence there is a unique injective map ip.r —* n for which 

= {i G J | V« = 0} e But then g = ^ n (h,c^ oy .. . ,<$._,)) in K&> because 
A(0 G AT„ for every i G J. 

To see that this decomposition is extremal, let also g = S5?(Ai,c?«*, . . • »< ! £?fo_i)) 
with an injective map t/>i : rj — » n. Then there exists some Ji G ^" so that the left hand 
side in 

&n (MO, 4"o)' • • • ' 1)) = &n ( h I (0. ^(0)' • • • ' ^(r. - 1)) 

is an extremal decomposition of g(i) in AT, and the equality itself holds for every /' G J\. 
Since 7i ^ 0, there exists a unique 8: r — ♦ n with \p\ 06 = tp and, because J| G JF, the 
decomposition g = F„(h, c^ 0) , cQ_ , } ) of g is extremal. This proves (2). 

For (3), we need that an extremal decomposition g = ST„(h, cg^, . . . , c^,,) in A^ 
of any g G A^ is its extremal decomposition also in L ( f\ This follows easily from the 
fact that (4) holds for K and L. 

To prove (4), let /o,...,/,,-, G £{*>, g G L^, and let C = (^)Wo.. . . ,/„-.) 
be the composite in L i!F) . Ug = * (n) , then C = * (m) because * (n) is a constant. Any 
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g G L n ^ \ {* (n) } belongs to by (1), and hence has an extremal decomposition 
g = S£(/», c$ 0) , . . . , cj^_i)). by (2). lff m = *< m > for somey € r, then C = *<"> follows 
from the definition of (5^,)* on the ultrapower and the fact that * (m) is constant. 
Otherwise also G KS^ for all j G «, by (1), and hence there exists some Ji G f such 
that C(/) = SJ, (g(iVo(0. • • • 1 (0) * for all ieJ 2 . But then C = Sr m (g,f 0t ...,/„_,) 
in \ and this completes the proof of (4) for KS^ ) and m 

4.5. LEMMA. LetE A :A^A* and E B : B —* B* be clone algebra embeddings satis- 
fying (1H4) of Lemma 4.4. Then 

A~B if and only ifA*~B*. 



Proof. To simplify the notation, let us suppose again that the embeddings £,< , £s are 
inclusions. We let (S^)a, (S%,)a , 0S£,)s> (S^Yb stand for the non-nullary clone operations of 
A,A*,B,B* respectively, and for the miliary operations of each of these four algebras. 

Let O: A — » B be a clone algebra isomorphism. Then <I> maps each sort A„ of A 
bijectively onto the sort B„ of B in such a way that O^"*) = cj n) for every j € n. 
Furthermore, if g = (S^M^c^, . . . , c { $ r _ X) ) is an extremal decomposition of some 
g G A n then <D(g) = (Sr n ) B (<P(h), c { $ 0) , . . . , cJJ._ 1} ) is an extremal decomposition of <D(g). 

From (1) we recall that A*=A n U {* (n) } and B* = B n U {* (fl) }, and set 

Wir x_ f«fe) forallgG^ 

It is clear that ¥ maps each A* bijectively onto B*. To prove that ¥ is an isomorphism, 
we only need to show that 

(Q ^((s^/o,..., /„_,)) = (a^m-.'w-i)) 

for any choice of/b, • • • € and g G A*. 
First suppose that g = * ( " } . 

Here(^)^,/ 0 ,...,/„-,) = * (m) and (S» m ) B (V(g),V(f 0 ), . . . ,V(f n __ y j) = *<*>, from 
(4) applied to both /I* and 5*. But then (C) follows because ¥(*<"•>) = *<">. 
Let g ^ *W next. 

Thenghasan extremal decomposition g = (^(h,^...^^) with /i ^ *W, 
and hence V(g) = (B^CIX^tr-- is an extremal decomposition of ^(g). 
If/vKO = * (m) for some / G r, then ¥(fo 0 ) = *< m) , and (C) follows from an application of 
(4) to both A* and B\ Otherwise/^,-, G A„ for all / G r by (1), and (C) obtains from (4) 
because <D is an isomorphism of A onto B. Therefore *F: /T -+ fl* is an isomorphism, as 
claimed. 

Conversely, assume y.A* — » B* to be an isomorphism of these clone algebras. If/ G 
A n then -ac(/) by (1), and hence -ac(T(/)) because -ac(;t< n >) is a first order formula 
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and V is an isomorphism. But then ¥(/) e B n by ( 1) again and, therefore, the restriction 
O of 4* to A maps A isomorphically onto B. m 

4.6 . Now we are ready to complete the proof of Proposition in 4. 1 , which claims that 
for any two infinite disciplined metrizable spaces^, X 2 with no absolute fixpoints and 
any k € {1 ...,oo}, 

C\o k (X { ) W C\o k (X 2 ) if and only if C\o k (\(Xi)) « Clo*(A(^ 2 ))- 

But this follows immediately from 4.4-5 and the fact that, similarly to monosorted al- 
gebras (see [4] for instance), heterogeneous algebras of the same type are elementarily 
equivalent if and only if they have isomorphic prime limits. 

4.7 . In 1 .9, we noted the following result. 

THEOREM [7]. Ifm,n,p£ { 1, 2, . . . , u} satisfy m <n < p, then there exist infinite 
metrizable spaces X\ andX 2 such that 

m = sup{/ < u | C\ 0i (Xi) = Clo,(* 2 )}, 
n = sup{/ < lj | Clo,(X,) ~ Clo,(*2)}, 

p = sup{/ < u | Clo,^,) « Clo,(^2)}. ■ 

Thus to demonstrate the claim of Theorem 1 .9 about the three clone segments, in view 
of Propositions 2.9, 3.5 and 4.1 we need only show that the spaces A'i andA^ of Theorem 
[7] may be chosen to be disciplined and without absolute fixpoints. This we do in the next 
section. 

5. Disciplined spaces with no absolute fixpoints abound. 

5.1 . Let I = U^ 0 1„ be a finitary type of (mono-sorted) universal algebras in which 
I„ denotes the set of all w-ary operation symbols. For a e E, we write ar a = n whenever 
a 6 Z„, and always assume that I 0 is infinite. 

Let P = (P, {p a | a £ Z}) be an absolutely free I-algebra over the empty set of 
generators. It is well-known that P = U*to^*» with 

Po - far | <r€Zo} and/Vi = P k U\J{p a (Pf) \ a € I \Io}, 

where the unions are disjoint and every p a with o G I \ S 0 is one-to-one. 

5.2. For any integer m > 1, let T(xo, ... ,Jc m -i ) denote the term algebra of type £ over the 
set {jco, . . . ,Jc m _i }. Then T(xo, • • • ,*«-i) is an absolutely free I-algebra freely generated 
by the set {x 0 , . . . ,x m -\ }, see [4] or [6]. 

We aim to define a mapping g (m) of T(jr 0 , • • • , *«-i) into the set P*" of all maps P™ -* 
P, where P is the underlying set of the 0-generated absolutely free I-algebra P from 5.1. 
We write ttJ" 0 to denote the i-tfa product projection P m -tP and, for each a £ lo, we let 
t/ m) stand for the constant map P" — » P with the value p a G P 0 . 
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We define a mapping g (m) recursively as follows. 

g^Xxj) = for I em, 
g (m \o) = (/ m) for a 6 So. and 
Q im) (a(to, .... t n -ij) =p a o (g (m) (t 0 )x • • • Xf> (w) (/„ , )) for ael n with n > 1 . 

It is easily verified that g (m) : J(x 0 , . . . ,x m -\) — » P** is one-to-one. 

For any given Q C Z \ (Z 0 U Zi ), let T n (x 0 , . . . ,x m -i) consist of all those members of 
T(jt 0 , • • • , *m- 0 which have a subterm 

a'(x io , . . . ,Xt mtf _ l ) with a' G ft and distinct x, 0 , . . . ,x ima ,_ , G {x 0 , . . . , }. 

Denoted = g (m) (l(x 0 , . . . ,x m _,)) and//™) = ^(T n (x 0 ,...,x m -,)). 
Now we are ready to state the main result of [7]. 

M [7]. If card Z 0 > 2*° + card(Z \ Z 0 ), then for every ft C Z \ (Z 0 U Z i ) there exists a 
metric r n on the set P so that, for the metric space X n = (P, r n ) and any integer m > 1, 
a mapping f : A£| — ► Ah is continuous exactly when f G /^ m) \ f/jj 0 . ■ 

In this section we prove that the spaces Xq constructed in [7] are disciplined, and that 
they have no absolute fixpoints when Z \ Zo ^ 0. To do this, we need to use some of 
their structural properties that were stated only implicitly in [7] or had only an auxiliary 
role there. 

5.3 . As in 3.4 of [7], we write Pq = {p a \ o G Zo}, B a = p„(P") for a G Z„ with n > 1, 

andB = \J{B a |<7GZ\Zo} = P\P 0 . 

In 3.5 of [7] it is shown that (P,r n ) is a metric space of diameter 1 such that 
(a) B is a closed subset of (P, r^), 

(3) Pa is an isometry (P",^) — * {B ay r a \B a ) when <r G Z„ \ ft, 

00 Tn(B a ,B a >) = 1 whenever <r, o J G Z \ Zo are distinct, 

(6) p a is an isometry (P", f£) — > (fl ff , t q \B a ) when cr G Z„ H ft, 
where f£ in (6) is an auxiliary metric which we describe in 5.4 below. 

From (a) and (7) it follows that every set B a with a G Z \ Z 0 is closed in X n , and 
implies that p a is a homeomorphism of A£ onto the closed subset B a for any <r G Z„ \ ft. 

5.4 . For P as in 5. 1 and B as in 5.3, let us write 

FUJI = to, • ■ • G P" | z, = z,} for distinct /J G n, 
P"[i, c] = {(z 0 , . . . ,z„_, ) G P" | z, = c} for / G « and c G P, 
P"[/, B] = {(z 0 , . . . ,z„_.) G P" | z, G 5} for / G «. 

and call these sets small subsets of AJj. It is clear that P"[iJ] and P"[i,c] are closed 
subsets of A£>, and that P"[/, #] is a closed subset of A£> because of 5.3(a). 

We need to show that, on any small subset of A?,, the product metric t£ induces the 
same topology as an auxiliary metric f£ from [7]. The latter metric is given for n > 2 in 
3.4 of [7] by 

r^y) = min{ 1 , 7^{x,y) + \g(x) - g(y)\ }, 
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for a real-valued function g: P" -» [0, 1 ] and a point a (n) = (ag°, . . . , a™ , ) G P" 0 satisfy- 
ing a* n) ^ aj n) whenever /,y G n are distinct. 

The function g, the point a (n) , and certain small subsets of Aq are related as follows: 

(1) g is continuous on X^ \ {a (n *}, 

(2) g(x) = 0 for any x G Ui^U^fh I 1 G w}, and also for any x outside a neigh- 
bourhood of a (n) which is disjoint with the sets \J{P"[i,j] \ i ^ jJJ G n} and 

IK^B.*] I * € »}, 

(3) g is discontinuous at a w . 

Therefore the restriction ofg to any small subset of A£> — including the sets P"[i, a { " ] ] — 
is continuous, so that f& and t£ induce the same topology on every small subset of A£. 
This implies that the identity map of A£ = (P 1 , t£) onto (/>", f£) sends each small subset 
homeomorphically onto a closed subset of (F\f£). Since each B a with a € X \ Z 0 is 
closed, 5.3(5) implies that, for any a G Z„ Pi Q, the restriction of p CT to any small subset 
of A^j is a homeomorphism onto a closed subset of Xq. 

5.5 . Here is a summary of earlier observations which will be needed in Proposition 5.6 
and in Section 7: 

(1) if a G Z„ \ Q and n > 1 , then p a is a homeomorphism of A?> onto a closed subset 
of Ah, 

(2) if a G Z„ n Q, then p ff is not continuous, but its restriction to any small subset of 
A£ is a homeomorphism onto a closed subset of A" n , 

(3) the topology of (P",f^) is finer than the topology of (P",r^) for every n > 2. 

5 .6. PROPOSITION. 77ie space Xq is disciplined for every Q C Z \ (Zo U Z , ). 

Proof. Write X = Xq. Since X is metrizable, we need only show that, for any 
m > 1 , every non-constant continuous map /: X" — ► X is of the form / = go 71^, where 
i^ M) :X m — ► X M is the projection associated with some A/ C m, and g is a homeomor- 
phism of A* 1 onto a closed subset of X. 

Let m > 1 and let/: A™ — -> A" be non-constant and continuous. According to 5.2, there 
is a unique term G T(x 0 , . . . ,**-i) \ ln(x 0 , . . . ,x m ^) with ^ m \t f ) =f. 

We use the term recursivity of the absolutely free Z-algebra T(jc 0 , . . . ,x m ..\ ) as follows. 

If t f = or, with i G m, then / = 7r< m) is the i-th product projection X™ —+ X, and hence 
it has a required decomposition, namely / = id* o7rj m) . Otherwise t f = o<<b, . . . , t n -\ ) for 
some a G Zand/ 0 ,...,/„-i G T(Ar 0 ,...,JC m _i)\T n (Afo,...,Jc m _i). Since/is non-constant, 
we must have a G Z„ with n > 1, and/ = p a o (/" 0 x • • • x/„_i), where / = Q (m \tj) for 
all y G « and at least one of the maps /: A 7 " — » X is non-constant. Assuming that 

for every y G «, there is some Mj C /w and some homeomorphism g, of X M ' onto a 
closed subset of A" such that fj = g, o *W>, 

(where we write Mj = 0, 7r <A ^ ) : X" — > A* and g,: A 0 — » A" when / is a constant map), we 
shall construct a required decomposition / = g o 7r (W) . 

First of all, we note that g = U{gj 1 7 G n) is a homeomorphism of n{A^ | y G fl} 
onto a closed subset of A". Set M = U{Af, | j G «}, and let 0 y : A/ y — A/ denote the 



Copyrighted material 



1318 



V. TRNKOVA AND J. SICHLER 



inclusion map. For any </? G X M write h(<p) = (^ot/i 0 <^o^_,). Then 

h.x M ^X\{x M ' \jen} 

is a homeomorphism of A*' onto a closed subset of n{A^ \j G n}. 

If A: A" — > (AT is the 'diagonal map' defined by A(y>) = (<£, . . . , <p) for all ^GT, 
and if x = Yl{^ Mj) \ j 6 «}, then tt o A = h o ti<">. Since/ = / 0 x • • • x has a 
decomposition 

it follows that / ={goh)o n (M) , where g o h is a homeomorphism of X M onto a closed 
subset of A 71 . 

For « > 1 and trGZ„\fi, the mapping p a is a homeomorphism of X" onto a closed 
subset of X, by 5.5(1), so that the composite g = p a o g o h is a. homeomorphism of 
X* 4 onto a closed subset of X again. Therefore / = p a of = go ^ has a required 
decomposition. 

Let a G ZnQ.Thena G Z„nQ for some n > 2, and we show thatf(X m ) is contained in 
a small subset of X" . Indeed, if at least one of the component maps f off = fo x • • • xf„- 1 
is the constant with the value c, then /(A™) C P"[j, c]. Assume that no map f with ;Gn 
is constant. If one of them, say f jo , is not a product projection, then f 0 = p„> o h for 
some /rA"" — > A" r<7 ' and </ G Z \ Z 0 , and hence /(/*") C P"[/o,B]. In the remaining 
case, every f is a projection nr 1 ': A 7 " — ► X, so that /(jco, • • • ,x M -i) = (*/„»••• >*'«-■)• 
Should x io , . . . ,*,„_, be pairwise distinct, then a(x io , . . . ,*/„_,) G T n (*o, • • • ,* m -i) and 
hence / would not be continuous. Therefore i r = i s for some distinct r,s G n, and hence 
/(/"") C P»[i r , /,]. Altogether,/^) is always contained in a small subset of A 71 . Since 
/(/*") is closed in A 71 , it is also closed in any small subset containing it. By 5.5(2), the 
restriction of p a to /(/"") maps the set /(/*") = g/KA**) homeomorphically onto a closed 
subset of X, so that g = p a ogo/risa homeomorphism of X M onto a closed subset of X 
and / = Pa°f = g° also in this case. 

Therefore every continuous non-constant /: A 7 " — > X with m > 1 has a required 
decomposition. ■ 

5.7. Lemma. If I. \ Z 0 ^ 0 ^ C Z \ (Z 0 U Zi ) is arbitrary, then there exists a 
continuous map s:X n —* X n such that sof f for every /: A^ — + Ah with k > 0. In 
particular, the space An has no absolute fixpoints. 

PROOF. If a G Z„ and /i > 1, then either n = 1 and a £ Q, or else n > 2 and 
the diagonal of A£> is a closed subset of the small subset /"'[O, 1] C A^. Hence s(x) = 
p a (x, ... ,x) is a homeomorphism of A'n onto a closed subset of B a for any a G Z \ Zo. 
If 5(a) = a for some a e P then <r(a, . . . , a) = a in the absolutely free 0-generated Z- 
algebra P, which is impossible. Therefore 50/ ^ / for every continuous/: Ajj — ♦ An 
with k>0. m 

5.8. Corollary. //Z \ Z 0 ^ 0 and n C Z \ (Zo U Z,), //ren ^ n « a disciplined 
space without absolute fixpoints. m 
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6. Operation clones of traces. 

6. 1 . To complete the proof of Theorem 1 .9, we recall that the trace A(A^ = (Q, ZU {■}) 
of any infinite metrizable space X = (P, t) has a binary operation • and a countable set 
Z = {7} U {7T y I j G w} of unary operations on the set Q = SUPU {A}, in which S 
is the collection of all one-to-one sequences in P. One can easily verify that the binary 
operation • and the set Z of the unary operations satisfy these identities: 

(1) (x -y) -z = x • (y • z) and* • y = y x 

(2) a{x y) = x • y for all a G Z, 

(3) pa(x) = a(x) for all p, a G Z, 

(4) a(x) ■ a(x) = a(x) for all a G Z, 

(5) Mx)-ir l (x)-y = x y, 

(6) nj(x)-n f (x)t{x) = 7r 0 (x)-7r,(x) for ally ^/ inw, provided = *, or t(x) = o(x) 
for some a e Z, or /(*) is missing altogether. 

6.2. Proposition. Let X = (P, /) be an infinite metrizable space with no isolated 
points. Then an identity holds in h(X) if and only if it is a consequence of the identities 
described in OH® above. 

Proof. For n,n' > l , let t(x\ , . . . , x„) and r'Ot', , . . . , jc^, ) be terms in the basic opera- 
tions of A(X) over the variables indicated. We need to show that any identity 

(I) /(*!,..., x„) = t , (x\,...,x'„>) 

either fails to hold in &(X) or else follows from (l)-(6). 

The following easily established properties of \(X) will be used: 

(a) if /(jci, . . . ,jc m ) is a term and p G P, then t(p, ...,p)=p, 

(b) if q,q' G Q and q ■ q' ^ A, then q = q' = q q' eP, 

(c) if r(xi, . . . ,x„) is a term and (^i, . . . ,q n ) G Q 1 with q ( = A for some i= I,...,*, 
then/(<7i,...,0,,) = A. 

We claim that an identity (I) is satisfied in fk(X) only when it is regular, that is, when 
it has the form 

(R) f (xi,..., *„) = ^(X\,...,X H ). 

If not, then, say, x\ $ {*',,.. . ,x' n ,}. Select any p ^ p' in P and substitute p for xi and p' 
for all other variables occurring in (I). Then t'ip', . . . ,p') = p' and t(p,p', . . .p') G {p, A}, 
by (a), (b) and (c). Since p' $ {p,A}, such an identity fails to hold in any H(X). 

Before discussing reasons why (R) holds or fails in H(X), we note that any term r in 
which • does not occur is either a variable or t(x) = a{x) holds in A(^) for some 0 G Z, 
by (3). i 

Case 1 . Suppose that • occurs on neither side of (R). Since members of Z act dif- 
ferently from one another and from the identity map on the trace MX) of any infinite X, 
the identity (R) holds in A(*) exactly when it follows from (l)-(6). 

We turn to the remaining possibilities now. 
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Suppose that • does occur in t(x\,. . . ,*„). Then (l)-(3) imply that / = t\ ■ . . . • t r where 
each /, is either one of the variables of /, or /,(*) = a(x) holds in MX) for some a G Z. 
Having applied (5) twice to each /, which is a variable*, we find that /,(x) = no(x) • n\(x) 
holds in h(X), and may henceforth assume that every /, has the form f,(x) = a(x) for 
some a G Z. Then, using (4) to eliminate all repetitions, we may also assume that /, ^ f, 
whenever i ^ / Having applied (6), we also conclude that each variable x needs to occur 
in / at most twice, and that t(x) = t x (x) if / is unary or else t{x) = t x (x) • T x — where the 
variables of T x are the variables of / other than .r, and t x (x) is one of the unary terms 

(U) 7T0(X) • 7T, (X), 7(X), Or 1tj{x) ■ 1{X\ Ttj(x) Wlthj G D 

in either case. 

In particular, for any unary term t in which • does occur, in any h(X) we have either 
t(x) = 7r 0 (jf) • tti(x) or t(x) = ttj(x) • l(x) for some j G CJ. 

For a future use we note that, since an infinite disciplined metrizable space has no 
isolated points, for any p G P there exists a one-to-one sequence s converging to p. In 
particular, there are sequences s, sj G S converging top such that ttj{s) ^ p for all j G u>, 
and TT k (sj) = p exactly when k = j G uj. In algebraic terms, this respectively means that 
ttj(s) ■ l(s) = A for all / G w, and that ir k (sj) • 7(s) ^ A if and only if k =j G w. 

Case 2. Suppose that • occurs in / but not in t '. Thus t'ix) = x or t'ix) = a(x) for 
some a G Z. We claim that (R) fails to hold in any MX). In the first case, t(s) G PU{ A} $ 
s = t'is) for any s G S. In the second, we choose a convergent sequence s G S so that 
7r,(s) ^ 7(5) for all j G w. Then t(s) = X <£ P and = <t(s) G P for every a G Z. 
Whence (R) fails to hold in A(X) in either case. 

CASE 3 . Suppose that • occurs in both / and d . Let the identity (R) hold in MX), and 
let x be one of its variables. Then t = t x (x) ■ T x and t! = tf x (x) ■ T x — where x occurs neither 
in T x nor in T' x and t x (x\ ^(x) are amongst the unary terms listed in (U). 

(3a) Suppose that t x (x) = a(x) for some a G Z. To show that t x (x) = ^(x), we select 
an s G 5 so that 7(s) G P and 7(s) ^ 7r,(.s) for all j G u>, and then substitute s for x 
and <7(s) for each of the remaining variables in (R). Then er(s) G P and, if T x and are 
present, also T x = V x = a(s), by (a). If (R) is a unary identity, then a(s) = ^(s). If not, 
then a(s) = t = ^(s) • a(s) by (4), and hence a(s) = ^(s) follows again, this time from 
(b). Since s is one-to-one and 7(s) ^ tt/(s), we have ir 0 (s) • n\(s) = A = 7ry(s) • 7(s) for 
all j G u;, and also a'is) ^ a(s) in P whenever a, a 1 G Z are distinct. Since 4(jc) is one 
of the terms listed in (U), the only remaining possibility is that ^{x) = cr(x). But then 
t x (x) = /((.r), as claimed. 

(3b) To show that / t (jc) = ^(jt) in all cases, suppose now that ^(jc) = Kjfec) ■ 7(jc) for 
some j G ut. Select s, G S so that 7(5, ) = 7r y (jy), and substitute Sj for jc and Wj{Sj) G P 
for all other variables in (R). Then t x (sj) = iTjisj) G P, and T x = T x = 79(5/)— if these 
terms are present. Then 79(5,) = £(jr y ) follows as in (3a), from where we also already 
know that 4(x) cannot be a{x) for any a G Z. If <(jc) = 7r 0 (*) • tti(x), then ^(5,) = A £ P 
because 5, is one-to-one, and (R) fails. The remaining possibility given by (U) is that 
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4(jc) = n k (x) • 7(x) for some k G u. But ^(sj) = ir k (sj) • 7(s,) = nisj) • tTj{sj) G P 
only when k=j because of (b) and the fact that sj is one-to-one. Therefore t x (x) = 
whenever neither term is equal to no(x) ■ 7Ti(x) in l\(X). This concludes the proof that 
t.x(x) = ^(x) follows for every variable jc of any identity (R) which holds in A(A r ). 

Altogether, any identity (I) either follows from (1H6) or else fails to hold in every 
h(X). u 

The proof of Theorem 1 .9 is now complete. 

7. Consequences and modifications. 

7. 1 . Let (30 U) be a concrete category with finite products preserved by its forgetful 
functor U. Then the clone Clo(a) exists for every object a of % with non-empty under- 
lying set. If b is another such object, then the suprema 

i(a,b) = sup{*+ 1 | C\o k {a) ~ C\o k {b)} and 
e(a,b) = sup{£+ 1 | Clo*(a) « Clo*(fc)} 

exist and their values belong to the set { 1, 2, ... , oo}. It is clear that the two resulting 
functions satisfy 

(1) <f(a,a) = oo, 

(2) (f(a, b) = ip(b, a), and 

(3) <p(b,c) > min{tp(a,b\ip(a,c)}. 

This leads to a natural question about the representability of ip by i or e over some 
category % of universal algebras. The question is addressed in Theorems 7.2-3 below. 
These follow from Theorems 1 and 2 of [7] — now valid for infinite disciplined metrizable 
spaces without absolute fixpoints — by means of Propositions 3.5 and 4. 1 . 

Remark. For the reciprocal p = 1 / tp of a function v? satisfying ( 1 H3) we have 

p(a, a) = 0, p(a, b) = p(b, a) and p(6, c) < max{p(a, b), p(a, c)}, 

which are the axioms of a non-archimedean pseudometric. Every metrizable 0-dimen- 
sional topological space can be metrized by a non-archimedean metric p with values in 
the set of integer reciprocals, see [2]. Theorem 7.2 below implies that any such pseu- 
dometric space can be represented by the reciprocal 1 / /' of our function i on a suitable 
collection of universal algebras. 

7.2. THEOREM. Let C be a set, and let <p:C x C — ► {1,2,... ,00} be a function 
satisfying 7.1(l}-(3). Then there exists a system {A c \ c e C} of finitary algebras with 
pairwise isomorphic clones of operations, and such that i(A a ,Ab) = l p( a ->b)f or every 
(a,b) e C x C. Moreover, if <p(a,b) > 1 for all (a,b) € C, then all algebras A c can be 
constructed over the same underlying set. ■ 

Since there are only countably many formulas in the first order language of clones and 
their ^-segments, a result by Erdos and Rado [3] implies that an analogue of Theorem 7.2 
for elementary equivalence cannot hold when cardC > 2**°. In Theorem 7.3 below we 
present a partial positive result that leaves open the remaining case of < cardC < 2*V 



Copyrighted material 



1322 



V. TRNKOVA AND J. SICHLER 



7.3. THEOREM. Let Cbea countable set, and let (p: C x C — ♦ { 1 , 2, . . . , oo} be a 
function satisfying 7.1(I)-(3). Then there is a system {B c | c G C} of finitary algebras 
with pairwise isomorphic clones of operations, and such that e{B a ,B} ) ) = <p(a,b) for 
every (a,b) G C x C. Moreover, if <p(a,b) > 1 for all (a,b) G C, then all algebras B c 



7.4. Concluding remarks. Other conclusions can be easily obtained by various 
manipulations of the algebraic trace h(X). 

(A) Theorems 7.2, 7.3 and 1 .9 — without their claims about operation clone isomorph- 
ism — hold also for algebras of other similarity types. 

(A 1 ) Countably many unary operations suffice. To see this, for any infinite disci- 
plined metrizable space X = (P, t) with no absolute fixpoints, we simply replace the 
trace A(A) = (£?,ZU {•}) by an algebra B(X) = (Q x Q, {o 2 \ o G Z} U {3 t P 0t Pi }) 
in which, for all (*o,*i) £ Q x Q, we set o 2 (xq,x\) = (a(xo),o(x\j) when a € Z, 
0(xq,x\) = (jco • jci.jco * *i) Pi(xo,x\) = (Xi,Xj) for /' 6 2. Then B(X) is a unary al- 
gebra such that g: B(Xf —* B(X) is a homomorphism exactly when g = f x / for some 
homomorphism / : A(Ay* — » &(X). 

( A2) A single u>-ary operation suffices as well. For any infinite disciplined metrizable 
space X = (P, t) with no absolute fixpoints, we replace A(A) by an algebra C(X) = (/?, O 
of type {u} in which R = PU{\} and, for any s G 



It is clear that, for n > 0, any homeomorphism h of A 71 onto a closed subset of X extends 
to a homomorphism £(A): C(A7" — » C(X), and that is constant whenever h is. Propo- 
sition 2.3 is, in fact, somewhat easier to prove for C(X) than for A(*), and the remainder 
is straightforward. 

(A3) In a forthcoming paper, it will be shown that three unary operations are also 
sufficient. The proof uses a modification of a type-reducing construction from [5]. 

(B) Theorem 1 .9 and its unary forms hold also for an algebra and its reduct— in place 
of two algebras with isomorphic operation clones. 

The spaces X x and X 2 used to prove Theorem 1 in [7], of which Theorem 1.9 is an 
algebraic translation by means of their traces have the same underlying set and 
satisfy Clo(*,) C C\o(X 2 ), see also 5.5(3). The algebras A(*,) and A(X 2 ) inherit these 
properties. Hence A(^i) can be expanded to an algebra 0(^0 by adding a new unary 
operation 8\ defined by 6\(x) = 7 2 (x), where 7 2 was the unary 'convergence' operation 
of MX 2 ). Then Oo(D(Xj )) = Clo(A(*, )) follows from Proposition 2.3 and the fact that 
Clo(*,) C C\o(X 2 ). The reduct D(X 2 ) = /k(X 2 ) is then obtained from D(*,) by removing 
the original 'convergence' operation 7 1 of A(A'i). 



can be constructed over the same underlying set. 



ns) = { 



p if j G f* - converges to p 
X in all other cases. 
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THE REPRESENTATION RING 
OF THE TWISTED QUANTUM DOUBLE 
OF A FINITE GROUP 

S. J. WITHERSPOON 



Abstract. We provide an isomorphism between the Grothendieck ring of mod- 
ules of the twisted quantum double of a finite group, and a product of centres of 
twisted group algebras of centralizer subgroups. It follows that this Grothendieck ring 
is semisimple. Another consequence is a formula for the characters of this ring in terms 
of representations of twisted group algebras of centralizer subgroups. 

Introduction. Let G be a finite group and k an algebraically closed field. Given a 
3-cocycle ur.G x G x G — » k x , there is a quasi-Hopf algebra D^iG) with underlying 
vector space (kG)* <S>k kG, where (kG)* is the Hopf algebra dual to the group algebra 
kG [1, 4, 7]. As an algebra, ir(G) is a crossed product of the Hopf algebras (kG)* and 
kG. There is also a construction of this algebra from the ordinary quantum double D(G), 
originally defined by Drinfel'd [9], analogous to that of a twisted group algebra from a 
group algebra. We have therefore chosen to call D^(G) the twisted quantum double of 
G. 

In the case k ■ C, it is indicated by Dijkgraaf, Pasquier, and Roche in [7] that there 
is a connection between the representations of D^(G) and conformal field theory; in 
this context, tensor products of these representations are of interest. Applications of 
such representations to generalized Thompson series and Moonshine are given in [1] 
by Bantay. In [14], Mason discusses a conjectured equivalence of categories between 
Z>"(G)-modules and modules of a certain vertex operator algebra, and in [8], Dong and 
Mason prove a special case of this conjecture. 

The algebra D^(G) turns out to be an example of a quasitriangular quasi-Hopf algebra. 
For our purposes, this implies that there is a representation ring R{[y(G)) which is both 
associative and commutative. 1 ne sum and product ot this ring are given by direct sum 
and tensor product of representations. In this exposition, we will analyze the structure 
of this ring, or more precisely of the Grothendieck ring ^.(D^G)), the quotient of 
R(D^(G)) by the ideal of short exact sequences. 

In Section 1 , we give an explicit definition ofZ>"(G), and discuss some of its properties. 
In particular, we show that CT(G) is semisimple if and only if the characteristic p of 
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k does not divide the order of G. In the non-semisimple case, the Grothendieck ring 
is a proper quotient of the representation ring, and is easier to study. In Section 1 we 
also state a characterization of indecomposable /^((TJ-modules, which is a simultaneous 
generalization of the case k = C in [7], and of the ordinary quantum double D(G) over 
an arbitrary field in [14]. 

Our main result (Theorem 4.4) describes the structure of the Grothendieck ring 
^(Z>TO), and implies that it is semisimple. More precisely, we show that there is 
an isomorphism from ^.(/^(G)) to the product of centres of twisted group algebras of 
centralizer subgroups of G. As a consequence, we give its set of characters, that is algebra 
homomorphisms from ^(£^((7)) to C. These characters distinguish modules up to their 
composition factors, and are analogous to Brauer characters of the group, which corre- 
spond to the characters of the Grothendieck ring 1<SkG) of the group algebra kG [2]. They 
are expressed in terms of characters of twisted group algebras of centralizer subgroups 
of G, that is trace functions of modules for these algebras. Such modules correspond to 
projective representations of the centralizer subgroups. Our theorem generalizes a result 
for the ordinary quantum double D(G) given in [16]. 

We first present our results in the special case k = C in Section 2. In this case the 
Grothendieck ring described above is equal to the representation ring, and proofs are 
straightforward. For the case of an arbitrary algebraically closed field k, we start with the 
same approach. However, now D^(G) is an algebra over k, whereas its representation 
ring is an algebra over C. The situation is complicated by the necessity of lifting certain 
function values from k to C. It also appears necessary to introduce a representation 
group, that is a covering group such that any projective representation (or representation 
of a twisted group algebra) may be lifted to an ordinary representation of this group. 
In Section 3, we present an analog of Brauer characters for twisted group algebras, and 
results relating these Brauer characters to those of representation groups. In Section 4, 
we use these results to prove our main theorem. 

All our modules will be finite dimensional over k, and all tensor products will be over 
k unless otherwise indicated. We caution that we will use the term "character" in two 
different ways. At times, it will refer to a homomorphism from an algebra to C. At other 
times, it will refer to the trace function of a module. In [3], Benson and Parker deal with 
this confusion by referring to the former as a "species." Here we hope that it will be clear 
from context which meaning is intended. 

1. Preliminaries. We will define the twisted quantum double as in [ 1 , 4, 7], although 
we do not require the underlying field to be C. Fix a finite group G and an algebraically 
closed field k of characteristic p. Let uj: G x G x G — ► k* be a 3-cocycle; that is 

u;(a, b, c)u<a, be, d)u>(b, c,d)= u(ab, c, d)u(a, b, cd) 

for all a<b,c,d € G. We assume that u) is normalized so that u{a,b,c) is equal to 1 
whenever one of a, b, or c is equal to the identity element 1 of G. The twisted quantum 
double E^iG) of G with respect to u is the following quasi-Hopf algebra with underlying 
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vector space (kG)* §t kG. (For the definition of a quasi-Hopf algebra, see [4] or [10].) 
Let {6 g <8> x} g>x eG be the canonical basis, where 8 g is the function dual to g £ G, so that 
bg(g) ~ 1 ar| d <M A ) = 0 when h € G, h f g. We will abbreviate 8 g ® * by <5 g .r. 
The product is given by 

MM = «t*A~9 « { J*'*** 

where Ofay) = ^^ffVy) . The element l^o = E^I is the multiplicative 
identity. We point out that this is the algebra structure given by the crossed product [15] 
of the Hopf algebras (kG)* and kG with respect to the conjugation action of G on (kG)* 
and the linear map a: kG®kG-^ (kG)* defined by 

afy ® y) = £ efay)8 g 

gee 

for all x,y EG. 

When we use the notation 8 g , we will mean S g l, and by x we will mean T. g eG^gX. 
Thus the elements {8 g } geG generate a subalgebra of D"(G) isomorphic to (kG)*, but the 
elements {x} xe c do not in general generate a subalgebra isomorphic to kG. However, 
the elements Jc are invertible, with JT 1 = Egec Ofa~ l ,x)- l 6gF*. 

The coproduct A: IT(G) — IT(G) <g> D*(G) is given by 

A(W = E UK h- x g){b h x) <g> ffajb 

h€G 

where l x (h, I) = ^''^"'1^ '^ . This coproduct is not coassociative in general, but 
quasi-coassociative; this means that there is an invertible element 

<£= E 

g,MeC 

in D^G)® 3 such that (A <8> id)A(a) = ^(idg^A^-' for all a e £T(G). 
A counit e and a coinverse s are given by 

e(8 g x) = and 5(6^) = 0 r i(x,*- V^fe,*" VS*-^'^ 

where <§ g( i is the Kronecker delta. These maps make the algebra D^(G) into a quasi-Hopf 
algebra [1,4, 7]. Verification of the details involves the following identities, which result 
from the 3-cocycle identity for uj: 

Q z (a, b%(ab, c) = 9 a -, 2a (b, c%(a, be), 
9 fa b%(a, bYt a (y, zYt b (f,f) = 9^(a, ^(y, z), 
Ifa byt z (ab, c)u(<f, b 2 , c 2 ) = l 2 (b, c)Tl 2 (a, bc)u(a, b, c), 

for all a, b, c,y,z € G. If u is trivial, we obtain the ordinary quantum double D(G) of 
G, which is a Hopf algebra. The quantum double construction was originally defined by 
Drinfel'd for any Hopf algebra [9]. 
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We will next define the representation ring. Let U and Vbe right D"(G)-modules. Then 
U <8> V is a right ZJ^GJ-module via the pullback of the natural action of D*(G) 8 D^G) 
onU®V with respect to the coproduct A: IT{G) IT{G) ® IT(G). This results in a 
right Z>"(G)-module since A is an algebra homomorphism. The field A: is a right Z^(G)- 
module via the pullback of the action of k on itself by right multiplication with respect 
to the counit c:Z> J (G) — + k. Up to isomorphism, this trivial module k is a multiplicative 
identity with respect to tensor product of modules; this follows from the counit property 
of a quasi-Hopf algebra. 

Let R(pP(G)) denote the representation ring of Er°(G), that is the C-algebra generated 
by isomorphism classes of finite dimensional right D^Q-modules with direct sum for 
addition and tensor product for multiplication. Then R(lT(G)) is a ring with identity 
given by the isomorphism class of the trivial module. Associativity of R^D^iG)) follows 
from the quasi-coassociativity of the coproduct for D^G). By abuse of language and 
notation, we will consider Z> J (G)-modules to be elements of the representation ring, 
when we really mean their isomorphism classes. 

We now define Ro (/^(G)) , the ideal of short exact sequences, to be the ideal generated 
by all U - If - V" where 0-+C/-*£/-*(/'-»0isa short exact sequence of IF{G)- 
modules. Notice that Ro (jy(G)} is in fact already generated by these elements as a vector 
space, since tensor products of modules are taken over the field k. The Grothendieck ring 
'RjjriGj) is defined to be the quotient of R(l> J (G)) by R 0 (Lr J (G)) . This C-algebra has 
a basis consisting of the images of the irreducible D^G^modules. 

The quasi-Hopf algebra D^G) is quasi triangular [4, 10], with 

R= £ * f I®**g and /T'= £ ^.(g^-'r'M^^i 37 - 

gMG gMG 

That is, RA(a)R~ l = a(A(a)) for all a £ D^(G), where a is the twist automorphism 
interchanging two factors. If U and V are Z>"(G)-modules, this equation yields an iso- 
morphism between the modules U ® Vand V® U, given by the twist a followed by the 
natural action of/?. Thus the representation ring ^(/^(G)) is commutative. 

We note that if (3: G x G — ► k* is a normalized 2-cochain, and 6(3 its coboundary 

8(3(a,b,c) = 0(b,cMa,bcWab,cr>(3(a,b)-\ 

then rr*0(<G) is isomorphic to IT(G) as an algebra. The map t: IT(G) -» Z>^(G) defined 
by 

is an algebra isomorphism. As a quasi-Hopf algebra, D^G) is obtained from Z>"(G) by 
the twist element 

gMG 
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That is, A(t(a)) = f(i ® ^A(a)))/^ 1 for all a G Z^(G) [4, 7]. The map i:IT(G) -> 
D^G) induces an isomorphism of representation rings 

t*:R(Lf*0(G)) -^R(IT(G)). 

We point out that // 3 (G,* X ) is finite of order dividing \G\p> (that part of \G\ not 
divisible by the characteristic p of k), and that any 3-cocycle uj is cohomologous to 
a 3-cocycle of finite order dividing \G\^. This can be shown by an argument exactly 
analogous to that for 2-cocycles in [6, Lemma 1 1.38]. In addition, a 3-cocycle of finite 
order cohomologous to uj may be chosen to be normalized as well. We will assume from 
now on that uj is normalized and of finite order dividing |G|^ . 

If jc commutes with g and h, then 

Vx(g, h) - l x (g, h) , 

uj{g,x,h) 

where these are the functions arising in the coproduct and product of D^{G) defined 
above. A calculation shows that this function 6 X : C(x) x C(x) — » k x is a 2-cocycle; that 
is Q x (ab, c)9 x (a, b) = 6 x (a, bc)6 x (b, c) for all a,b,ce C(x). 

As an algebra, D^{G) is semisimple if and only if the characteristic p of k does not 
divide the order of G; we outline an argument here. If p does not divide \G\, then both 
kG and (kG)* are semisimple, and Maschke's Theorem implies that D^(G) is semisimple 
(see [13]). On the other hand, IT(G)/ ker(c) and kt, where / = Zgechg, are both 
isomorphic to the trivial module k as /^(G^-modules. If IF(G) is semisimple, then k 
occurs only once in a decomposition of the right regular module Z?"(<7) into a direct sum 
of irreducible /^(Q-modules. In particular, any composition series of D^G) contains 
exactly one factor isomorphic with k. This implies that p does not divide | G\ , as otherwise 
we have / e ker(c). 

In the case k = C, a description of the irreducible D w (G)-modules is given in [7]; in 
case uj is trivial, a description of the indecomposable D^Q-modules over an arbitrary 
field k is given in [14]. These results are easily extended. The irreducible (respectively, 
indecomposable) Z> J (G)-modules are indexed by pairs (U, g) where g is a representative 
of a conjugacy class of G and U is an irreducible (respectively, indecomposable) A**C(g)- 
module. Here k 9 * C(g) denotes the twisted group algebra with basis {xf, }hedg) and product 
XhXt = 0 g (h, l)x hl . This characterization of Zr(G)-modules arises from an equivalence 
of categories, which we describe next, following the approach in [14]. It may be helpful 
to notice first that any /^(G^-module U is a direct sum of subspaces U = Exec U • &x, 
and the invertible elements g of ET(G) permute these subspaces, as b x g = g£> g -\ xg . 

For each x € G, there are subspaces IT(x) = Egec^g and S*{x) = H g£C (x)kb x g 
of iy{G). Note that S*(x) is a subalgebra, with identity element 8 Xt isomorphic to 
the twisted group algebra k°'C{x). Accordingly, we identify these two algebras in the 
following lemma. If K is a conjugacy class of elements of G, let Cy(K) = £*ejc D^ix). 
Then [y(G) ~ ® K £r(K) is a direct sum of two-sided ideals, where K ranges over all 
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conjugacy classes of G. Therefore an indecomposable /^(GJ-module is a module for 
precisely one of the D^AT). The irreducible (indecomposable) Z>(G)-modules are thus 
characterized by the equivalence of categories given in the following lemma, whose 
proof is straightforward. We point out that D^(x) is free as a left 5^(:c)-module, so the 
equivalence of categories in the lemma preserves short exact sequences. 

LEMMA 1.1. Let K be a conjugacy class of G, x G K, and Mod-A^ x C(x) and Mod- 
D^iK) the categories of right k° x C(x)- and iy(K)-modules, respectively. Let U G Mod- 
it*' C(jc) and V G Mod-IT(K). The maps 

U^U <g>su W D^jc) and V V ■ S x 

define an equivalence of categories between Mod-**' C(x) and Mod-Z^(AT), where D^{K) 
acts on V ®s*(x) ^(x) by right multiplication in the second factor. 

2. The case k = C. In this section, we present results about the representation ring 
R(D^(G)) in the special case k = C. These results will be generalized to an arbitrary 
algebraically closed field in Section 4. 

Certain characters of IT(G) are introduced in [ 1 ]; these are the functions from G x G to 
C defined for each D^CTJ-module U by sending a pair (h,g) to the trace Tr(<5/,g, U) of the 
action of Shg on U. With these characters, we will build certain functions from ^(/^(G)) 
to C which are in fact algebra homomorphisms, or characters of the representation ring. 

We first define, for each g G G, a linear function f g : R(lT(Gj) -» C"*C(g) by 

MU)= E W&uto 

for any Z>^(G)-module V. We will see that f g is an algebra homomorphism with image 
the centre Z(C fl «C(g)) of C°*C(g). In fact, the product of all f g , taken over a set of 
representatives g of conjugacy classes of G, provides an algebra isomorphism from 
R(IT(G)) to U g Z(C e *C(gj), as is proved in Theorem 2.2 below. 

LEMMA 2. 1 . The function f g : /?(ZF(G)) — ♦ C5C(g) is an algebra homomorphism. 

PROOF. It may be checked that/ g takes the trivial D^GJ-module C to the identity x, 
of C e *C(g). Now let U and V be /^(Q-modules. Then 

fgW g (V)= E e g (hJ)TT(8 h g,U)Tr(5 e g,V)x he 

h,l£C(g) 

= £ 0 g (h,h- ] e)Tr(6 h g,U)TT(6„ le g,V)x ( , 
MeCfc) 

where in the second sum, i has been replaced by h~ l t. 
On the other hand, 

f g (U®V)= E Ti0,f,t/®P)x,. 
teog) 
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The action of 6 ( g on U<g> V is given by A(6 ( g) = Ehec 7g(A, A 1 l)&hg®f>h-HS- Therefore 
MU® V) = £ 7 g (A, A"' OTifoS, L0Tr(£„-,g, >>, 

h,tec(g) 

+ E 7*(A, A-^Trfe */)Tr(W, ^ 

«€0(g),A6G-Cte) 

as 0 g is equal to l g on C(g) x C(g). Comparing the two calculations, we need only see that 
the second sum above is equal to zero. But when A C(g), the element b h g is nilpotent, 
so its trace on any module is zero. ■ 

We next show that the image of/ g is contained in Z(C*«C(g)). We first give a basis of 
Z(C°'C(g)). We say that A € C(g) is O g -regular if 0 g (A,c) = 9 g (c, A) for all c <E Cq^A). 
It may be checked that there is a basis of Z(C*«C(g)) indexed by conjugacy classes of 
0 g -regular elements A of C(g). Each basis element is a sum E a^x* over all elements W 
in the C(g)-conjugacy class of A, with coefficients satisfying = 9 g (h,y)/9 g (y, A*). 

Letting h,y G C(g), note that 

If U is any ZT(G)-module then, we have 

The second equality follows from the definition of 9, as A, y € C(g). The 2-cocycle 
property of 9 g on C(g) yields 

9 g (h,cy) 9 g (h,c)9 g (h,y) 
9 g (cy,hy) 0 g (c,A)0 g (y,A") 

whenever c e Cq^A). These equations imply that Tr(^g, U) = 0 if A is not 0 g -regular, 
and that/ g (t/) is in Z(C**C(g)). 

Next we will need to consider characters of a twisted group algebra C a H, where a is a 
2-cocycle on a finite group H. For facts about characters of finite dimensional algebras, we 
refer to [6, Section 9B], and note that C a H is semisimple [11]. Specifically, a character is 
a function from CZ/to C given by the trace function on a C a //-module. These functions 
are not algebra homomorphisms in general. The character of an arbitrary C a //-module is 
the sum of the characters of its irreducible direct summands. The irreducible characters, 
that is characters corresponding to irreducible modules, are linearly independent over C. 

We will consider the vector space generated by C°//-modules with direct sum for 
addition. By the trace function of an element of this space, we will mean the corresponding 
linear combination of trace functions of C°//-modules. 
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THEOREM 2.2. The product n of the maps f g induces an isomorphism of algebras 

R(!T(G))-^nz{C e 'C(g)), 

8 

the product taken over a set of representatives g ofconjugacy classes ofG. In particular, 
the representation ring R(D^(G)) is semisimple. 

Proof. Lemma 2. 1 and the discussion following it show that each f g is an algebra 
homomorphism with image contained in Z(C°*C(g)j. It remains to prove that n is a 
bijection. Let a € ^(/^(G)) with n(a) = 0. Fix x e G and let g € C(x). Then f g (a) = 0 
implies that Tr(8 x g,a) = 0, where we have extended the trace function linearly to 
/^/^(G)). But Tr(5 x g, a) = Tr(6 x g, ab x ), where here we may consider ab x as an element 
of the vector space generated by C 5 ' C(jc)-modules via the equivalence of categories given 
in Lemma 1 . 1 . In other words, we have Tr(x g , ab x ) = 0 for all g e C{x). As the irreducible 
characters of C?*C(x) are linearly independent, a6 x = 0 in the vector space generated by 
C <> 'C(jc)-modules. But this is true for all x € G. As Zy(G)-modules are determined by 
their jc-components, we have a = 0. Thus we have shown that n is injective. 

To complete the proof, we must show that tt is also surjective, which will follow 
once we see that the dimensions of R(lT(Gj) and n g Z(C*«C(g)) are the same. By 
Lemma 1 . 1 , the dimension of R(D^(Gfj is equal to the sum, over a set of representatives 
g of conjugacy classes of G, of the number of irreducible C fl «C(g)-modules. This is also 
the dimension of U g Z(C B 'C{g)). m 

Now we may write down a table consisting of the values of the complete set of 
characters of the semisimple algebra R(lT(G)) on the irreducible Z^(G)-modules. Such 
a character maps a ^(Q-module U to 

-L £ Tift* ^pm, 

degPAecoj) 

where g e G and p is an irreducible character of C e *C{g)\ that is, p is the trace function of 
an irreducible C*«C(g)-module. This table is analogous to the character table of a finite 
group. 

3. Brauer characters. Here we present an analog of Brauer characters for a twisted 
group algebra, and some results about their connection to Brauer characters of a repre- 



Let H be a finite group, k an algebraically closed field of characteristic p, and a : H x 
H—>k x a 2-cocycle. As before, we denote by k°H the twisted group algebra that has 
basis {x h } h£ff with multiplication given by XhX t = a(h, t]x h (. We will assume that a is 
normalized so that x\ is the multiplicative identity of k°H, and also that a is of finite order 
m dividing \H\p> . A calculation shows that the elements Xf, of Ic^H all have finite order 
dividing m-\H\. The representations of k°H correspond to projective ^-representations 
of //with associated 2-cocycle a [1 1, 12]. 
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Let E be a representation group for H. That is, there is a central extension 

I — >A — >E^H — ► 1 

of H such that any projective ^-representation of H may be lifted to an ordinary it- 
representation of E which on a section of n is projectively equivalent to the original 
one [6, 12]. We will recall the explicit formula for so lifting a projective representation 
pu'.H — » GL(U), with associated 2-cocycle a, to an ordinary representation cjij-.E — > 
GL((/) [6, 1 1]. We will denote by JJJJ) the vector space U with the structure of a kE- 
module via o u . Fix a section {y g ) g at of tt with y\ = 1 , and \et a:H x H —> A be the 
corresponding 2-cocycle provided by multiplication in E. That is, yg h = a(g, h)y gh for 
all g, A £ H. Let A € A be the character such that 

\(a(g,h)) = a(g,h)p(gMh)p(ghr l 

for some function /x: (7 — > k* t and all g,h £ H. Then ou(ay g ) - \{a)p{g)pu(g) for 
all a € € //. Note that // and A depend only on a and a, and not on the given 
representation. 

Another way to construct a v is by using the isomorphism kE ~ flaer^tf, where 
T is a transversal of the group B 2 {H,k x ) of 2-coboundaries in the group Z 2 (H,k x ) of 
2-cocycles [12, Theorem 3.3.5]. The explicit map kE — » is given by sending ay g 
to A(a)/i(g)jrg, where A and /i are as above. A ^//-module may be lifted to kE via this 
surjection. Morphisms of ^//-modules then become morphisms of A:£-modules, and 
short exact sequences are preserved. 

Let r = The r-th roots of unity in k form a cyclic group of order r. For the rest 
of this section, fix an isomorphism between the group of r-th roots of unity in k and the 
group of r-th roots of unity in C. 

Let U be a k 0 //-module, ami ^ a p-regular element of H; that is, p does not divide the 
order of g. The element^ of k^H has finite order dividing r = so its eigenvalues 
on V are r-th roots of unity, and may be lifted to C via the isomorphism chosen above. 
Define the Brauer character of U to be the function from the set of p-regular elements 
of// to C given by sending g to the sum of the lifts of the eigenvalues of x g on U to C. In 
case a is the trivial 2-cocycle, this Brauer character is the usual one for H. For a given 
p-regular element g £ H, define the function t g .„ on k" //-modules by setting t g , a (U) 
equal to the Brauer character of U evaluated at g. If £ is a representation group of H, 
we will denote by t y (V)the Brauer character of a *£-module V evaluated at a ^-regular 
clement.y of E. 

LEMMA 3.1. Let V be a kfH-module, g a p-regular element of H, and L(U) the lift of 
U to a kE-module defined above. Then t Vg (L(U)) is a nonzero scalar multiple of t g , a (U). 
Further, the same nonzero scalar multiple is involved in any such lift, so that ty K o Lisa 
nonzero scalar multiple of t g , a - 
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Proof. It follows from the construction of the module HJJ) that the action of x g on 
U and that of v g on £((/) differ by the scalar /t(g). It may be checked that n(g) is an r-th 
root of unity, where r = |£|^. Lifting eigenvalues to C then, and summing, we obtain the 
desired result. ■ 

We will need to exploit the connection between induced ^-modules and induced 
/^//-modules. Let L be a subgroup of H, and consider a also to be a 2-cocycle of L by 
restriction. If U is a A^L-module, then U may be lifted to an ordinary kir~~ 1 (L)-module 
L(U) using the same procedure as for H and E (see [ 1 2, Lemma 3.3. 1 ]). 

LEMMA 3.2. Let U be a k^L-module and U k?H the corresponding induced 
kfH-module. Then L( U ®k a L k^H) ' s isomorphic to the induced kE-module L( U) <S>u kE, 
where J = ir~ l (L). 

PROOF. Let gi , . . . , g t be a transversal of L in H with g\ = 1 . Then y g} , . . . ,y gl is a 
transversal of J in E. Define a linear map from L(U) <S>u kE to UJU (g)^ Ic^H) by 

where we have identified the underlying vector spaces of U and L\U). It may be checked 
that this map is an isomorphism of fc£-modules. ■ 

Let g be a /^-regular element of H, and consider the function t gM on ^ a (g)-modules, by 
replacing H by (g) in the above development. We may extend / g , a linearly to the C- vector 
space generated by A a (g)-modules (with direct sum for addition). In the next section, we 
will use t g , a to define a function generalizing^ from Section 2. The following lemma 
will be used in the proof that such a function is an algebra homomorphism. 

LEMMA 3.3. Let g be a p-regular element of H, and L a proper subgroup of (g) . Then 
the kernel of t g , a contains any ^{gYmodule induced from k°L. 

PROOF. Let E be a representation group for (g). By Lemma 3.1, o£isa nonzero 
scalar multiple of t g , a . Therefore it suffices to show that ty t {UV®*L ^(g))) = 0 for any 
k° {.-module U. By Lemma 3.2, L\U ® M Jfc^g)) ~ L\U) ® u kE as *£-modules, where 
J - n'\L) is a proper subgroup of E containing A. But t yg factors through (y g ) < E; 
that is t Vt = t y o i^ t) , where t yg is the corresponding trace function on *(y g )-modules, 
and if } denotes restriction of modules [3]. Using |J to denote induction of modules, 
we apply to HJJ) \j= UV) ®u kE, and the Mackey Subgroup Theorem yields 

Now, as E is generated by y g and A, and J a is a proper subgroup of E containing A, we 
have^g £ J". Therefore/ 7 fl iy g ) is a proper subgroup of (y g ). Thus L{U) |5l^ ) is in 
the image of induction from proper subgroups of (y g ), and so is in the kernel of l yg [3]. ■ 

We return to the situation where u is a 3-cocycle of a finite group G, of finite order 
dividing \G\p> . The next lemma deals with a certain direct summand of the tensor product 

£/<g> Kof two Z>"(G>modules (/and V. 
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LEMMA 3.4. Let V and V be IT(G)-modules, i ? G G, and g G C(i). Consider 
{U <g) V)8 t to be a k°'(g)-module via Lemma 1.1 and restriction. Then the subspace 

of (U ® V)8 t is a k°'(g)-submodule, and is a sum of modules induced from proper 
subgroups of '(g). 

Proof. First note that the given subspace is indeed a ^'(g)-submodule, as the sum 
is over elements h not in C(g), and 8f,g = g8g-\h g . We may assume that U and V are 
indecomposable Z>"(G>modules. By Lemma 1 . 1 then, there are elements x,y G G such 
that 

U~U6 x ®s*wir(x) and V~V6 y ®9>{y) 
Consider a A^'(g)-submodule 

of Ehec-Qg) Wh ® Vb h -\ t , where the pair (x,y) ranges over all (g) -conjugates of 
(h 0 , ho 1 1) for a fixed h 0 E G - C(g). The (g)-conjugates of (ho^h^i) form an or- 
bit as a (g)-set which is isomorphic to L \ (g) for a proper subgroup L of (g), as ho is not 
in C(g). It may be checked that M is induced from the A^'L-module V8^ <8> ^-i/i the 
map from (US* ® *\-t<) <8y a A*'(g) to A/ defined by 

(life, $ v6 h -> t )®8 t gj *-+ uS^gJ <g> v\-.,g>7g,(/»o, h o l O 

is a (g)-module isomorphism. ■ 

4. Characters of the Grothendieck ring. In this section, we will give a general- 
ization of Theorem 2.2 to an arbitrary algebraically closed field k of characteristic p. 
Let r be the square of the least common multiple of all \E\p>, where E ranges over a 
set of representation groups for the subgroups of G. For the rest of this section, fix an 
isomorphism between the r-th roots of unity in k and in C. 

Given g G G, the values of 9 g are |C/Ly-th roots of unity in k, by the restriction on the 
order of uj. Therefore, they may be lifted to C by our chosen isomorphism of r-th roots 
of unity in k and in C. Denote by & g : C(g) x C(g) — » C* the lift of the map 9 g with 
respect to this isomorphism, and note that 0 g is a 2-cocycle on C(g) as well, but with 
values in C x . Thus we may form the twisted group algebra C 0 *C(g). 

Now let g be a p-regular element of G. Define a linear function f g : R{D^(G)) — » 
C e *C(g)by 

fg(V) = £ WW*)**, 

where U8 h is considered to be a k°"C(h)-modu\e via Lemma 1 . 1 , and the functions t gSh 
are defined in Section 3. We will show that f g is an algebra homomorphism and that its 
image is contained in Z(C e «C(g)). 
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We point out that the proofs of results in Section 2 cannot be translated directly in this 
situation. The functions t g ^ h require looking at eigenvalues of invertible operators. By 
contrast, the proof of Lemma 2. 1 depended on considering traces of nilpotent operators. 

THEOREM 4. 1 . The function f g : R(D^{G)) — » C 0 *C(g) is an algebra homomorphism. 

PROOF. It may be checked that f g takes the trivial Z>^(G)-module k to the identity x\ 
of C e *C(g). Now let (/and Vbe /^(O-modules. Then 

h,l£C(g) 

Mecife) 

where in the second sum, we have replaced i by h~ x L On the other hand, 

UV<Z>V)= Y, t g ,o t {( u ®*V>t)x ( 

= E ^(Ew 4 ®^ t W 

rec(g) v /,eG ' 

Now is the sum of lifts of eigenvalues of 8[g on a module to C, and the action of 
8(g on a tensor product is given by 

Note that when A G C(g), Ub h ®V8 h ^ t isa^'(g)-submoduleof(Lr®F)6,,and7g(/j,/r l O 
lifts to e g (/«, A" 1 £) in C, since l g = 0 g on C(g) x Cfe). Therefore 

tteQg) 

♦ E <*•,( E Uh^raiHfW 

By Lemmas 3.3 and 3.4, the second summand above is zero, and so f g {U® V) is equal 

For the next lemma, we recall the description of Z(C e *C(g)) given in Section 2. It 
has a basis indexed by 0 g -regular elements h, each basis element being a sum £ a^x^ 
over all elements in the C(g)-conjugacy class of h, with coefficients satisfying a& = 
& g (h,y)/® g (y, hp). We note that by the construction of 0 g , h is ©^-regular if and only if 
h is 0 g -regular. 

LEMMA 4.2. The image of f g is contained in Z(C @ *C(gj). 
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Proof. Let U be a /^(O-module and h,y G C(g). As b»g = y ~\%^&hg)y, the 
action of aY^l Shg on Ubf, corresponds to the action of b^g on Ub^ under the vector space 
isomorphism sending Ubh to Ubh> defined by right action by y. Thus the eigenvalues of 
b^g on Ub h v are times the eigenvalues of b h g on t/<5/,. So we have 

If /i is ©^-regular, this is what we needed to show, as §jgjj = by the definitions. 

If h is not ©^-regular, these equations force t gt9li (Ub h ) = 0 (see Section 2). ■ 

Finally, we will consider the Grothendieck ring ^(rr(G)), which is the quotient of 
R(IT(G)) by the ideal R 0 ([r{G)) of short exact sequences defined in Section 1. 

LEMMA 4.3. The ideal /? 0 is the kernel of the homomorphism 

ir.R{lT{G)) ^Y[Z(C**C(g)) 

8 

given by the product of the maps f g taken over a set of representatives g of conjugacy 
classes of p-regular elements of G. 

Proof. First let 0 — » £/ — ♦ {/—►£/' 0 be a short exact sequence of E^(G)- 
modules. Now, V = T. x eG US* as a direct sum of vector spaces, and similarly for 
the others. The Z>^(G)-module maps induce linear maps U'b x — » Ub x — > lf'b x , which 
correspond to maps of /^'C(jc)-modules by Lemma 1.1. That is, for each x G G, we have 
a short exact sequence of k 0, C(x)-modules 

0 — >lfb x — > Ub x —> U"b x —+0. 

Fix x G G and let g be a p-regular element of C(x). Let £ be a representation group of 
C(x), with E/A ~ C(x) for a central subgroup of E. We will use notation as in Section 3. 
We claim that tg^^Ubx — U'b x — U"b x ) = 0. Now, the above short exact sequence of 
A^'C(;c)-modules may be lifted, by the discussion at the beginning of Section 3, to a short 
exact sequence UfJb x — U'b x — U"b x ) of /rE-modules. By Lemma 3. 1 , t y o L is a nonzero 
scalar multiple of t gjBt . Any short exact sequence is in the kernel of t yg [3], so we do have 
t gA (Ub x - U'b x - U"b x ) = 0. Therefore, f g (U - V - U") = 0 for all p-regular elements 
g of G, and so R 0 (rr(G)) is contained in the kernel of tt. 

Now let b G R(rr(G)) with tt(6) = 0. Fix x G G and let g be a p-regular element of 
C(x). Then f g (b) = 0 implies that t g ,eXbb x ) = 0 for all jc G C(g), where bb x is considered 
to be an element of the vector space generated by ^C(jf)-modules. Lift bb x to E\ that 
is, lift all modules involved in bb x to E and form the corresponding linear combination 
Ubb x ). By Lemma 3.1, L(bb x ) is in the kernel of t yg for all p-regular elements g of C(x). 
Therefore L(bb x ) is in the kernel of t ayg for all elements ay g of E with g a p-regular 
element and a £ A, as a simply acts as the scalar A(a)/*(1) on any such module lifted 
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from C(x). But all p-regular elements of E are of this form. This implies that L(b6 x ) is in 
the ideal of short exact sequences of ££-modules [2]. The short exact sequences which 
appear in L(bS x ) involve only &£"-modules which correspond to /fc^C(;c)-modules. Thus 
bb x is in the vector space generated by short exact sequences of k 6 ' C(jt)-modules, and this 
is true for all x e G. By the characterization of Z>"(G)-modules discussed in Section 1, 
each is determined by its .x-components, where x ranges over a set of representatives of 
conjugacy classes of G. And a short exact sequence of ^C(x)-modules corresponds to a 
short exact sequence of D^GJ-modules, as noted in the text preceding Lemma 1.1. Thus 
beR 0 (lT(G)). m 

THEOREM 4.4. The product it of the maps f g induces an algebra isomorphism 

H(ir(G)) ^nz(c e <c(g)), 
g 

the product taken over a set of representatives g of conjugacy classes of p-regular 
elements of G. In particular, the Grothendieck ring ^(Z>"(G)) is semisimple. 

PROOF. By Lemma 4.3, the homomorphism it induces an injection from ^(/^(G)) 
to ]\ g Z(C®*C{gj). It remains to prove that it is a surjection, which will follow once we 
see that the dimensions of these two finite dimensional algebras are the same. 

The dimension of the Grothendieck ring ^(/^(G)) is equal to the number of irre- 
ducible D^G^-modules. By the characterization given in Lemma 1 . 1 , this is 

Y, (number of irreducible ^'C(jc)-modules), 

where the sum is taken over a set of representatives x of conjugacy classes in G. But 
the number of irreducible A^'C(x)-modules is the number of conjugacy classes in C(x) of 
p-regular {^-regular elements [5, 12]. 

On the other hand, the dimension of U g Z(C e *C(g)) is equal to 

£ (number of 0 g -regular conjugacy classes in C(g)), 
g 

the sum taken over a set of representatives g of conjugacy classes of p-regular elements 
inG. 

But these quantities are just two different ways of counting the orbits in the G-set 

{(x,g) | x € G,g is a p-regular element of G,xg = gx, and* is ©^-regular}, 

where G acts by conjugation on each factor. This follows from the observation that x is 
a ©^-regular element of C(g) if and only if g is a ^-regular element of C(x). To see this, 
note that when g and x commute, 

Og(x,c) _ 6 x (c,g) 
6 g (c,x) O x (g,c) 
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whenever c G C{x) n C(g). ■ 

The theorem allows us to write down a table consisting of the values of the complete 
set of characters of the semisimple ring ^(/^(G)) on the images of the irreducible 
/^(t^-modules. Such a character maps the image of a Z^(C)-module U to 

ae 8PhQC{g) 

where g € G and p is an irreducible character of C e «C(g). These characters therefore 
distinguish Z>^(G)-modules up to their composition factors, and this table of characters 
is analogous to the Brauer character table of a finite group. 
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